The importance and need for system monitoring and analysis in HPC operations and research

06/13/2018
by   Florina M. Ciorba, et al.
0

In this work, system monitoring and analysis are discussed in terms of their significance and benefits for operations and research in the field of high-performance computing (HPC). HPC systems deliver unique insights to computational scientists from different disciplines. It is argued that research in HPC is also computational in nature, given the massive amounts of monitoring data collected at various levels of an HPC system. The vision of a comprehensive system model developed based on holistic monitoring and analysis is also presented. The goal and expected outcome of such a model is an improved understanding of the intricate interactions between today's software and hardware, and their diverse usage patterns. The associated modeling, monitoring, and analysis challenges are reviewed and discussed. The envisioned comprehensive system model will provide the ability to design future systems that are better understood before use, easier to maintain and monitor, more efficient, more reliable, and, therefore, more productive. The paper is concluded with a number of recommendations towards realizing the envisioned system model.

READ FULL TEXT

page 2

page 7

research
09/25/2019

MPCDF HPC Performance Monitoring System: Enabling Insight via Job-Specific Analysis

This paper reports on the design and implementation of the HPC performan...
research
06/18/2019

From Facility to Application Sensor Data: Modular, Continuous and Holistic Monitoring with DCDB

Today's HPC installations are highly-complex systems, and their complexi...
research
01/29/2018

A cost effective and reliable environment monitoring system for HPC applications

We present a slow control system to gather all relevant environment info...
research
08/02/2023

PROV-IO+: A Cross-Platform Provenance Framework for Scientific Data on HPC Systems

Data provenance, or data lineage, describes the life cycle of data. In s...
research
09/26/2018

dynamicMF: A Matrix Factorization Approach to Monitor Resource Usage in High Performance Computing Systems

High performance computing (HPC) facilities consist of a large number of...
research
02/24/2019

Sustaining Research Software: an SC18 Panel

Many science advances have been possible thanks to the use of research s...
research
06/22/2023

Sustainable HPC: Modeling, Characterization, and Implications of Carbon Footprint in Modern HPC Systems

The rapid growth in demand for HPC systems has led to a rise in energy c...

Please sign up or login with your details

Forgot password? Click here to reset