Fixed-PSNR Lossy Compression for Scientific Data

05/17/2018
by   DingDingwen Tao, et al.
0

Error-controlled lossy compression has been studied for years because of extremely large volumes of data being produced by today's scientific simulations. None of existing lossy compressors, however, allow users to fix the peak signal-to-noise ratio (PSNR) during compression, although PSNR has been considered as one of the most significant indicators to assess compression quality. In this paper, we propose a novel technique providing a fixed-PSNR lossy compression for scientific data sets. We implement our proposed method based on the SZ lossy compression framework and release the code as an open-source toolkit. We evaluate our fixed-PSNR compressor on three real-world high-performance computing data sets. Experiments show that our solution has a high accuracy in controlling PSNR, with an average deviation of 0.1 5.0 dB on the tested data sets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/17/2020

FRaZ: A Generic High-Fidelity Fixed-Ratio Lossy Compression Framework for Scientific Floating-point Data

With ever-increasing volumes of scientific floating-point data being pro...
research
06/23/2018

Optimizing Lossy Compression Rate-Distortion from Automatic Online Selection between SZ and ZFP

With ever-increasing volumes of scientific data produced by HPC applicat...
research
05/15/2023

Black-Box Statistical Prediction of Lossy Compression Ratios for Scientific Data

Lossy compressors are increasingly adopted in scientific research, tackl...
research
07/09/2023

Hierarchical Autoencoder-based Lossy Compression for Large-scale High-resolution Scientific Data

Lossy compression has become an important technique to reduce data size ...
research
06/22/2022

ROIBIN-SZ: Fast and Science-Preserving Compression for Serial Crystallography

Crystallography is the leading technique to study atomic structures of p...
research
12/06/2017

A posteriori noise estimation in variable data sets

Most physical data sets contain a stochastic contribution produced by me...
research
06/12/2017

Z-checker: A Framework for Assessing Lossy Compression of Scientific Data

Because of vast volume of data being produced by today's scientific simu...

Please sign up or login with your details

Forgot password? Click here to reset