Enabling Reproducible Analysis of Complex Workflows on the Edge-to-Cloud Continuum

09/03/2021
by   Daniel Rosendo, et al.
0

Distributed digital infrastructures for computation and analytics are now evolving towards an interconnected ecosystem allowing complex applications to be executed from IoT Edge devices to the HPC Cloud (aka the Computing Continuum, the Digital Continuum, or the Transcontinuum). Understanding end-to-end performance in such a complex continuum is challenging. This breaks down to reconciling many, typically contradicting application requirements and constraints with low-level infrastructure design choices. One important challenge is to accurately reproduce relevant behaviors of a given application workflow and representative settings of the physical infrastructure underlying this complex continuum. We introduce a rigorous methodology for such a process and validate it through E2Clab. It is the first platform to support the complete experimental cycle across the Computing Continuum: deployment, analysis, optimization. Preliminary results with real-life use cases show that E2Clab allows one to understand and improve performance, by correlating it to the parameter settings, the resource usage and the specifics of the underlying infrastructure.

READ FULL TEXT

page 1

page 2

research
08/04/2021

Reproducible Performance Optimization of Complex Applications on the Edge-to-Cloud Continuum

In more and more application areas, we are witnessing the emergence of c...
research
07/24/2023

KheOps: Cost-effective Repeatability, Reproducibility, and Replicability of Edge-to-Cloud Experiments

Distributed infrastructures for computation and analytics are now evolvi...
research
04/29/2022

Distributed intelligence on the Edge-to-Cloud Continuum: A systematic literature review

The explosion of data volumes generated by an increasing number of appli...
research
07/20/2023

ProvLight: Efficient Workflow Provenance Capture on the Edge-to-Cloud Continuum

Modern scientific workflows require hybrid infrastructures combining num...
research
05/15/2023

Validity Constraints for Data Analysis Workflows

Porting a scientific data analysis workflow (DAW) to a cluster infrastru...
research
04/05/2022

Compute- and Data-Intensive Networks: The Key to the Metaverse

The worlds of computing, communication, and storage have for a long time...
research
12/13/2022

Automated Cache for Container Executables

Linux container technologies such as Docker and Singularity offer encaps...

Please sign up or login with your details

Forgot password? Click here to reset