DeepAI AI Chat
Log In Sign Up

The Ghost of Performance Reproducibility Past

08/27/2022
by   Srinivasan Ramesh, et al.
University of Oregon
Brookhaven National Laboratory
0

The importance of ensemble computing is well established. However, executing ensembles at scale introduces interesting performance fluctuations that have not been well investigated. In this paper, we trace our experience uncovering performance fluctuations of ensemble applications (primarily constituting a workflow of GROMACS tasks), and unsuccessful attempts, so far, at trying to discern the underlying cause(s) of performance fluctuations. Is the failure to discern the causative or contributing factors a failure of capability? Or imagination? Do the fluctuations have their genesis in some inscrutable aspect of the system or software? Does it warrant a fundamental reassessment and rethinking of how we assume and conceptualize performance reproducibility? Answers to these questions are not straightforward, nor are they immediate or obvious. We conclude with a discussion about the performance of ensemble applications and ruminate over the implications for how we define and measure application performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/09/2022

Reproducibility in Optimization: Theoretical Framework and Limits

We initiate a formal study of reproducibility in optimization. We define...
10/26/2021

Diversity and Generalization in Neural Network Ensembles

Ensembles are widely used in machine learning and, usually, provide stat...
04/20/2018

Use Cases of Computational Reproducibility for Scientific Workflows at Exascale

We propose an approach for improved reproducibility that includes captur...
03/15/2022

Reproducibility and Performance: Why Choose?

Research processes often rely on high-performance computing (HPC), but H...
01/12/2018

SwarmRob: A Toolkit for Reproducibility and Sharing of Experimental Artifacts in Robotics Research

Due to the complexity of robotics, the reproducibility of results and ex...
04/30/2021

Forming Ensembles at Runtime: A Machine Learning Approach

Smart system applications (SSAs) built on top of cyber-physical and soci...
08/30/2022

Distributed Ensembles of Reinforcement Learning Agents for Electricity Control

Deep Reinforcement Learning (or just "RL") is gaining popularity for ind...