DeepAI AI Chat
Log In Sign Up

Correlation-wise Smoothing: Lightweight Knowledge Extraction for HPC Monitoring Data

by   Alessio Netti, et al.

Modern High-Performance Computing (HPC) and data center operators rely more and more on data analytics techniques to improve the efficiency and reliability of their operations. They employ models that ingest time-series monitoring sensor data and transform it into actionable knowledge for system tuning: a process known as Operational Data Analytics (ODA). However, monitoring data has a high dimensionality, is hardware-dependent and difficult to interpret. This, coupled with the strict requirements of ODA, makes most traditional data mining methods impractical and in turn renders this type of data cumbersome to process. Most current ODA solutions use ad-hoc processing methods that are not generic, are sensible to the sensors' features and are not fit for visualization. In this paper we propose a novel method, called Correlation-wise Smoothing (CS), to extract descriptive signatures from time-series monitoring data in a generic and lightweight way. Our CS method exploits correlations between data dimensions to form groups and produces image-like signatures that can be easily manipulated, visualized and compared. We evaluate the CS method on HPC-ODA, a collection of datasets that we release with this work, and show that it leads to the same performance as most state-of-the-art methods while producing signatures that are up to ten times smaller and up to ten times faster, while gaining visualizability, portability across systems and clear scaling properties.


page 1

page 5

page 7

page 9


DCDB Wintermute: Enabling Online and Holistic Operational Data Analytics on HPC Systems

The complexity of today's HPC systems increases as we move closer to the...

Operational Data Analytics in Practice: Experiences from Design to Deployment in Production HPC Environments

As HPC systems grow in complexity, efficient and manageable operation is...

Programming at Exascale: Challenges and Innovations

Supercomputers become faster as hardware and software technologies conti...

Modularis: Modular Data Analytics for Hardware, Software, and Platform Heterogeneity

Today's data analytics displays an overwhelming diversity along many dim...