Understanding GPU-Based Lossy Compression for Extreme-Scale Cosmological Simulations

04/01/2020
by   Sian Jin, et al.
0

To help understand our universe better, researchers and scientists currently run extreme-scale cosmology simulations on leadership supercomputers. However, such simulations can generate large amounts of scientific data, which often result in expensive costs in data associated with data movement and storage. Lossy compression techniques have become attractive because they significantly reduce data size and can maintain high data fidelity for post-analysis. In this paper, we propose to use GPU-based lossy compression for extreme-scale cosmological simulations. Our contributions are threefold: (1) we implement multiple GPU-based lossy compressors to our opensource compression benchmark and analysis framework named Foresight; (2) we use Foresight to comprehensively evaluate the practicality of using GPU-based lossy compression on two real-world extreme-scale cosmology simulations, namely HACC and Nyx, based on a series of assessment metrics; and (3) we develop a general optimization guideline on how to determine the best-fit configurations for different lossy compressors and cosmological simulations. Experiments show that GPU-based lossy compression can provide necessary accuracy on post-analysis for cosmological simulations and high compression ratio of 5 15x on the tested datasets, as well as much higher compression and decompression throughput than CPU-based compressors.

READ FULL TEXT

page 1

page 3

page 9

research
04/01/2021

Adaptive Configuration of In Situ Lossy Compression for Cosmology Simulations via Fine-Grained Rate-Quality Modeling

Extreme-scale cosmological simulations have been widely used by today's ...
research
07/19/2020

cuSZ: An Efficient GPU-Based Error-Bounded Lossy Compression Framework for Scientific Data

Error-bounded lossy compression is a state-of-the-art data reduction tec...
research
11/10/2017

In-Depth Exploration of Single-Snapshot Lossy Compression Techniques for N-Body Simulations

In situ lossy compression allowing user-controlled data loss can signifi...
research
12/21/2022

Scalable Hybrid Learning Techniques for Scientific Data Compression

Data compression is becoming critical for storing scientific data becaus...
research
05/26/2021

Scalable Multigrid-based Hierarchical Scientific Data Refactoring on GPUs

Rapid growth in scientific data and a widening gap between computational...
research
06/10/2022

PILC: Practical Image Lossless Compression with an End-to-end GPU Oriented Neural Framework

Generative model based image lossless compression algorithms have seen a...
research
06/12/2017

Z-checker: A Framework for Assessing Lossy Compression of Scientific Data

Because of vast volume of data being produced by today's scientific simu...

Please sign up or login with your details

Forgot password? Click here to reset