Elastic Compute Clusters on EGI Federated Cloud for

Elastic Compute Clusters on EGI Federated Cloud for Computational Biodiversity Ignacio Blanquer 1, Alain Franc 2, Jean. Marc Frigerio 2, Miguel Caballer 1 LIFEWATCH-CC UPV– I 3 M 2 INRA 1 www. egi. eu EGI-Engage is co-funded by the Horizon 2020 Framework Programme of the European Union under grant number 654142

10/15/2021 2

10/15/2021 3

Which Tools are available? Alignment (multiple) Phylogeny Distance based methods Smith-Waterman (exact) Evolution models Graph clustering MDS - NLM Aggregative clustering (fast , , ) 10/15/2021 4

Molecular based taxonomy & systematics: An open route for (new) methods Sequences known by pairwise distances Clustering Distance geometry pattern recognition machine learning Multidimensional Scaling linear and nonlinear (e. g. Sammon, 1969) Manifold learning Iso. Map, Eigen. Map, etc … Graph based methods spectral clustering Machine learning SVM, … 10/15/2021 5

Galaxy Portal 10/15/2021 6

Where is it possible to compute? • Local Galaxy server From a unique portal the Galaxy instance • Mesocentre (Tier 2) Avakas 1000 cores • Tier 1 (IDRIS, one pipeline, not via Galaxy) • EGI GRID France-Grille Where from? • Cloud (next slides …) From any computer connected to internet Currenty available from French Guiana (IP Cayenne works with it) 10/15/2021 7

Need for a Elastic Compute Cluster • Usage of external resources enables users to have own resources and skip other user’s queuing time – Higher predictability of the response time. – Potentially shorter response time. • Cloud-based back-ends require management – Booting and shutting down resources. – Re-configuring clusters. – No management leads to resource waste. • Working node configuration can be tailored to users’ need – Reduced cost selecting rightmost configuration. 10/15/2021 8

Software Architecture End. User’s Web Browser Front/ End Contextu alized WM Images Galaxy portal IM Service VMI Repository Basic WN Image INRA’s Site 10/15/2021 Other Images UPV’s EGI Fed. Cloud Site 9

Technologies Used • Galaxy (galaxyproject. org) – Bioinformatics portal that transforms operations into Jobs submitted to a back-end batch queue. • Infrastructure Manager (IM www. grycap. upv. es/im) – Platform-agnostic, dynamic configuration service that enables re-configuring a cluster. • Elastic Compute Cloud Clusters (EC 3 www. grycap. upv. es/ec 3) – Automatic elasticity service based on batch queue length. 10/15/2021 10

(Re)-Configuration • Infrastructure Manager configures VMs from recipes • It can restart a broken configuration or rearrange the changes on an active configuration. • It Works with a plethora of CMPs, including OCCI and EGI Federated Cloud. 10/15/2021 11

Elastic Compute Cloud Cluster • EC 3 (Elastic Cloud Computing Cluster) leverages CLUES to create elastic virtual clusters in the Cloud. • It also uses Infrastructure Manager (IM) for the reconfiguration. • It provides automatic elasticity based on the batch queue workload. 10/15/2021 12

10/15/2021 13

10/15/2021 14

Conclusions • Elastic clusters can be used as a back-end for batch-oriented processing services. • Provides users with their own infrastructure – Homogeneous response time. – Fewer side-effects. • Cluster is deployed using end-users’ credentials and resource consumption is limited to the minimum. • Cooling down and stabilization rules reduce overheads. 10/15/2021 Insert footer here 15