The EUBra-BIGSEA enables applications to effectively scale across the infrastructure, providing also to the developers appropriate abstractions to specify QoS constraints and a unified programming interface that includes computing, data analytics, and security APIs.
EUBra-BIGSEA develops a programming layer for Big Data to transparently build applications composed of data operators mapped to different Big Data frameworks. The benefits of the QoS cloud infrastructure services are not limited to Big Data applications but support any heterogeneous workload.
The base of this programming layer is the COMPSs, a programming framework that aims to facilitate the parallelisation of existing applications. For that purpose, it offers a simple programming model based on sequential development in which the user is mainly responsible for (i) identifying the functions to be executed as asynchronous parallel tasks and (ii) annotating them with annotations or standard Python decorators.
A runtime system is in charge of exploiting the inherent concurrency of the code, automatically detecting and enforcing the data dependencies between tasks and spawning these tasks to the available resources and provide scalability and elasticity features allowing the dynamic provision of resources.
COMPSs is already capable of orchestrating applications that process big amounts of data and facilitates the access to data as much as possible.
EUBra-BIGSEA extends COMPS baseline to include:
- A data management layer of the programming model with the support to the data models, to the security hints and to the QoS constraints.
- A runtime that supports the interoperability data layer and the deployment services.
- A support for generalized reductions (important in many of the Big Data analytics algorithms), both at the application programming interface as well as run-time support.