Universally Quantized Neural Compression

06/17/2020
by   Eirikur Agustsson, et al.
7

A popular approach to learning encoders for lossy compression is to use additive uniform noise during training as a differentiable approximation to test-time quantization. We demonstrate that a uniform noise channel can also be implemented at test time using universal quantization (Ziv, 1985). This allows us to eliminate the mismatch between training and test phases while maintaining a completely differentiable loss function. Implementing the uniform noise channel is a special case of a more general problem to communicate a sample, which we prove is computationally hard if we do not make assumptions about its distribution. However, the uniform special case is efficient as well as easy to implement and thus of great interest from a practical point of view. Finally, we show that quantization can be obtained as a limiting case of a soft quantizer applied to the uniform noise channel, bridging compression with and without quantization.

READ FULL TEXT

page 16

page 17

page 18

page 19

page 20

page 21

research
12/05/2016

Towards the Limit of Network Quantization

Network quantization is one of network compression techniques to reduce ...
research
05/11/2023

Vector Quantization with Error Uniformly Distributed over an Arbitrary Set

For uniform scalar quantization, the error distribution is approximately...
research
12/11/2022

Error-aware Quantization through Noise Tempering

Quantization has become a predominant approach for model compression, en...
research
08/16/2021

Distance-aware Quantization

We address the problem of network quantization, that is, reducing bit-wi...
research
10/25/2021

Demystifying and Generalizing BinaryConnect

BinaryConnect (BC) and its many variations have become the de facto stan...
research
07/12/2021

HEMP: High-order Entropy Minimization for neural network comPression

We formulate the entropy of a quantized artificial neural network as a d...
research
06/27/2023

On the reconstruction of bandlimited signals from random samples quantized via noise-shaping

Noise-shaping quantization techniques are widely used for converting ban...

Please sign up or login with your details

Forgot password? Click here to reset