EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

EUROPE - BRAZIL COLLABORATION OF BIG DATA SCIENTIFIC RESEARCH THROUGH CLOUD-CENTRIC APPLICATIONS

Press [ esc ] or close+

EUBra-BIGSEA performance guarantee for Big Data applications

Performance Guarantee_EUBraBIGSEA

About

This asset is fully developed and integrated in the framework of the EUBra-BIGSEA project and it is based on the combination of three key components:

1) EC3 which automates the deployment and the initial configuration of a Big Data application and provides also mechanisms for runtime re-configuration (developed by Universitat Politecnica de Valencia),

2) a rule-based module for pro-active run-time policies specification and execution (developed by Federal University of Campina Grande),

3) a module implementing optimization based policies able to identify the deployment configuration of minimum costs that provides also performance guarantees (e.g., jobs are executed within a time limit or data streams are executed with no loss at a given rate, developed by Politecnico di Milano and Federal University of Minas Gerais).

 

Features, Benefits & Downloads

This asset allows to run Big Data application providing a priori performance guarantees. Since the general trend is to run Big Data applications in cloud environments where resource contention is significant, providing a solution that is able to cope with such issues is of paramount importance. The sets of components and services provided will be able to consider more information about the application in order to enable a smarter initial configuration for periodic jobs and a fast reaction time for the case of disturbances during the execution.

 

  • The three components are developed separately at the four Institutions (UPV, UFCG, Polimi and UFMG). EC3 is more mature and partially in production (basic case). The three components will be integrated during the second year of the project.
  • The open source license scheme is still under definition. Most of the components will rely on Apache 2.0, unless the dependencies to other libraries introduced in the implementation are not compatible with Apache 2.0.
  • The modules are being developed considering an OpenStack cloud environment and the usage of OpenStack Monasca as a monitoring tool.

 

Users

Currently, these three components are used within academic environments and are developed and tested within lab deployments.

 

Papers & References

  • A Predictive Approach for Enhancing Resource Utilization in PaaS Clouds. Henrique Truta, José Vivas, Andrey Brito, Telles Nobrega. The 32nd Annual ACM Symposium on Applied Computing. Proceedings of the 2017 ACM Symposium on Applied Computing