Elastic Cloud Computing Cluster (EC3) is a tool to create elastic virtual clusters on top of Infrastructure-as-a-Service (IaaS) providers, either public (such as Amazon Web Services, Google Cloud or Microsoft Azure) or on-premises (such as OpenNebula and OpenStack). EC3 offers recipes to deploy TORQUE (optionally with MAUI), SLURM and MESOS clusters that can be self- managed with CLUES. It starts with a single-node cluster and working nodes are dynamically deployed and provisioned to fit increasing load (number of jobs at the LRMS). Working nodes will be undeployed when they become idle. This introduces a cost-efficient approach for Cluster-based computing. The development has been performed completely by the I3M-UPV.
There are several tools in the market for creating self-managed virtual clusters. However, they rely on existing VMIs that the user has to maintain and customize. EC3 can be used on top of generic VMIs (e.g. an UBUNTU 14.0, a CENTOS 6.5, etc.) and reusing the same installation recipes in different cloud platforms with minimal changes. EC3 increases portability, interoperability and reduces vendor lock-in. Moreover, it uses Ansible for the configuration of the software, leveraging from the huge community and existing recipes.
Download and Installation
Current version is in production stage, expecting to have new improvements during the EUBra-BIGSEA lifetime to increase performance (self creation of base images), interoperability (TOSCA support), integration with other components of the project and the support of ahn extended catalogue.
EC3 Open source software released under the Apache 2.0 license.
EC3 is a lightweight service and it needs to be maintained to give support to the new versions of the external software supported.
It uses configuration recipes that can be stored in the Ansible Galaxy repository (not necessarily) or locally. It will also support shortly a local repository of components to facilitate the maintenance of the recipes.Extending the service with the recipes for the user own software requires a training in Ansible, yaml and similar DevOps concepts and tools.
EC3 is currently used in different scientific and industrial sectors, as it is a horizontal tool capable of deploying general purpose clusters. UPV has experiences of use in Computational Chemistry and Big Data Analytics, among others.