EVEREST: A design environment for extreme-scale big data analytics on heterogeneous platforms

03/06/2021
by   Christian Pilato, et al.
0

High-Performance Big Data Analytics (HPDA) applications are characterized by huge volumes of distributed and heterogeneous data that require efficient computation for knowledge extraction and decision making. Designers are moving towards a tight integration of computing systems combining HPC, Cloud, and IoT solutions with artificial intelligence (AI). Matching the application and data requirements with the characteristics of the underlying hardware is a key element to improve the predictions thanks to high performance and better use of resources. We present EVEREST, a novel H2020 project started on October 1st, 2020 that aims at developing a holistic environment for the co-design of HPDA applications on heterogeneous, distributed, and secure platforms. EVEREST focuses on programmability issues through a data-driven design approach, the use of hardware-accelerated AI, and an efficient runtime monitoring with virtualization support. In the different stages, EVEREST combines state-of-the-art programming models, emerging communication standards, and novel domain-specific extensions. We describe the EVEREST approach and the use cases that drive our research.

READ FULL TEXT

page 1

page 2

page 3

page 4

03/18/2020

Convergence of Artificial Intelligence and High Performance Computing on NSF-supported Cyberinfrastructure

Significant investments to upgrade or construct large-scale scientific f...
04/20/2022

Enabling Dynamic and Intelligent Workflows for HPC, Data Analytics, and AI Convergence

The evolution of High-Performance Computing (HPC) platforms enables the ...
04/05/2021

DataOps for Societal Intelligence: a Data Pipeline for Labor Market Skills Extraction and Matching

Big Data analytics supported by AI algorithms can support skills localiz...
09/16/2018

A Distributed Learning Architecture for Scientific Imaging Problems

Current trends in scientific imaging are challenged by the emerging need...
11/16/2020

ACCORDANT: A Domain Specific Model and DevOpsApproach for Big Data Analytics Architectures

Big data analytics (BDA) applications use machine learning algorithms to...
01/23/2018

Task-parallel Analysis of Molecular Dynamics Trajectories

Different frameworks for implementing parallel data analytics applicatio...
09/30/2021

An Automated Scanning Transmission Electron Microscope Guided by Sparse Data Analytics

Artificial intelligence (AI) promises to reshape scientific inquiry and ...