EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

Press [ esc ] or close+

COMPSs - COMP Superscalar

COMPSs - EUBra-BIGSEA

 

About

COMPSs is a programming framework that aims to facilitate the parallelisation of existing applications written in Java, C/C++ and Python scripts. For that purpose, it offers a simple programming model based on sequential development in which the user is mainly responsible for identifying the functions to be executed as asynchronous parallel tasks. A runtime system is in charge of exploiting the inherent concurrency of the code, automatically detecting and enforcing the data dependencies between tasks and spawning these tasks to the available resources, which can be nodes in a cluster, clouds or grids. In cloud COMPSs provides scalability and elasticity features allowing the dynamic provision of resources.
The transparent deployment of COMPSs applications in the cloud is delegated to the PMES service that deals with the intricacies of deployment, contextualization operations and the installation of the application packages and required libraries, as well as to the monitoring of the process. A dashboard is also available for the configuration of the user cloud environment.
In a nutshell, the main added value of COMPSs is its focus on different capabilities in the same framework with a low learning curve as developers do not have to deal with application programming interfaces (APIs). What is more, the development and execution of the applications is not restricted to a proprietary infrastructure as interoperability is a key feature.

Features, Benefits & Downloads

Features & Benefits

Automatic parallelization of sequential code without the need to adopt any specific Application Programming Interface (API). Support to Java, C/C++ and Python. The same code can be executed transparently with regards to the underlying infrastructure.

Automatic scaling and elasticity features so the number of available resources can be adapted to the actual execution needs.

Interoperability with different cloud providers to run computational loads on multi cloud environments without the need of code adaptation.

Availability of tools that ease: the COMPSs applications implementation by means of an Integrated Development Environment (IDE); the application deployment in distributed infrastructures by means of the Programming Model Enactment Service (PMES); and the monitoring of executions by means of the Monitoring and Tracing tools.

 

Download & Installation

  • COMPSs and PMES are constantly maintained and updated. The software is available as install packages and source code. They are also available on the EGI Application Database (AppDB) for virtual appliances.
  • Open source and licensed under Apache 2.
  • The installation of the packages automatically resolves the dependencies.
  • COMPSs does not include any API, the idea is that the user code is optimized by the runtime. The user is only required to provide information on the tasks composing the application.
  • Downloads, tutorials and manuals are available http://compss.bsc.es

 

Users

COMPSs has been adopted and extended in many scientific projects offered as a tool to develop scientific applications and optimize their execution on distributed infrastructures such as VENUS-C (20 user communities), Optimis, EUBrazilOpenBio,EUBrazilCloudConnect and is now leveraged in European Grid Infrastructure (EGI) as enabling technology for the execution of composed workflows on the federated cloud testbed. In EGI Fed Cloud COMPSs has been adopted to implement services for the biodiversity community.
The COMPSs group is collaborating with the BSC Life Sciences department for the development of Virtual Research Environment (MuG) that aims to answer the needs of the emerging worldwide 3D/4D genomics community where the COMPSs framework is used to implement the computational services to provide transparent access to applications and genomic data.

 

Papers & References

  • PyCOMPSs: Parallel computational workflows in Python, Enric Tejedor, Yolanda Becerra, Guillem Alomar, Anna Queralt, Rosa M. Badia, Jordi Torres, Toni Cortes, Jesús Labarta,  IJHPCA 31(1): 66-82 (2017), DOI: 10.1177/1094342015594678