Soft-to-Hard Vector Quantization for End-to-End Learning Compressible Representations

04/03/2017
by   Eirikur Agustsson, et al.
0

We present a new approach to learn compressible representations in deep architectures with an end-to-end training strategy. Our method is based on a soft (continuous) relaxation of quantization and entropy, which we anneal to their discrete counterparts throughout training. We showcase this method for two challenging applications: Image compression and neural network compression. While these tasks have typically been approached with different methods, our soft-to-hard quantization approach gives results competitive with the state-of-the-art for both.

READ FULL TEXT
research
10/18/2021

Wideband and Entropy-Aware Deep Soft Bit Quantization

Deep learning has been recently applied to physical layer processing in ...
research
06/26/2006

Entropy And Vision

In vector quantization the number of vectors used to construct the codeb...
research
11/23/2017

In Defense of Product Quantization

Despite their widespread adoption, Product Quantization techniques were ...
research
02/13/2017

Soft Weight-Sharing for Neural Network Compression

The success of deep learning in numerous application domains created the...
research
04/07/2021

Learned transform compression with optimized entropy encoding

We consider the problem of learned transform compression where we learn ...
research
11/08/2014

Stacked Quantizers for Compositional Vector Compression

Recently, Babenko and Lempitsky introduced Additive Quantization (AQ), a...
research
08/26/2019

Differentiable Product Quantization for End-to-End Embedding Compression

Embedding layer is commonly used to map discrete symbols into continuous...

Please sign up or login with your details

Forgot password? Click here to reset