EVEREST: A design environment for extreme-scale big data analytics on heterogeneous platforms

by   Christian Pilato, et al.

High-Performance Big Data Analytics (HPDA) applications are characterized by huge volumes of distributed and heterogeneous data that require efficient computation for knowledge extraction and decision making. Designers are moving towards a tight integration of computing systems combining HPC, Cloud, and IoT solutions with artificial intelligence (AI). Matching the application and data requirements with the characteristics of the underlying hardware is a key element to improve the predictions thanks to high performance and better use of resources. We present EVEREST, a novel H2020 project started on October 1st, 2020 that aims at developing a holistic environment for the co-design of HPDA applications on heterogeneous, distributed, and secure platforms. EVEREST focuses on programmability issues through a data-driven design approach, the use of hardware-accelerated AI, and an efficient runtime monitoring with virtualization support. In the different stages, EVEREST combines state-of-the-art programming models, emerging communication standards, and novel domain-specific extensions. We describe the EVEREST approach and the use cases that drive our research.


page 1

page 2

page 3

page 4


Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure

Significant investments to upgrade or construct large-scale scientific f...

Enabling Dynamic and Intelligent Workflows for HPC, Data Analytics, and AI Convergence

The evolution of High-Performance Computing (HPC) platforms enables the ...

DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching

Big Data analytics supported by AI algorithms can support skills localiz...

A Distributed Learning Architecture for Scientific Imaging Problems

Current trends in scientific imaging are challenged by the emerging need...

ACCORDANT: A Domain Specific Model and DevOpsApproach for Big Data Analytics Architectures

Big data analytics (BDA) applications use machine learning algorithms to...

Task-parallel Analysis of Molecular Dynamics Trajectories

Different frameworks for implementing parallel data analytics applicatio...

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Artificial intelligence (AI) promises to reshape scientific inquiry and ...