DeepAI AI Chat
Log In Sign Up

End-to-end Learning of Compressible Features

by   Saurabh Singh, et al.

Pre-trained convolutional neural networks (CNNs) are powerful off-the-shelf feature generators and have been shown to perform very well on a variety of tasks. Unfortunately, the generated features are high dimensional and expensive to store: potentially hundreds of thousands of floats per example when processing videos. Traditional entropy based lossless compression methods are of little help as they do not yield desired level of compression, while general purpose lossy compression methods based on energy compaction (e.g. PCA followed by quantization and entropy coding) are sub-optimal, as they are not tuned to task specific objective. We propose a learned method that jointly optimizes for compressibility along with the task objective for learning the features. The plug-in nature of our method makes it straight-forward to integrate with any target objective and trade-off against compressibility. We present results on multiple benchmarks and demonstrate that our method produces features that are an order of magnitude more compressible, while having a regularization effect that leads to a consistent improvement in accuracy.


page 1

page 2

page 3

page 4


Improved Projection Learning for Lower Dimensional Feature Maps

The requirement to repeatedly move large feature maps off- and on-chip d...

End-to-End Optimized Speech Coding with Deep Neural Networks

Modern compression algorithms are often the result of laborious domain-s...

Feature-informed Embedding Space Regularization For Audio Classification

Feature representations derived from models pre-trained on large-scale d...

Model Compression by Entropy Penalized Reparameterization

We describe an end-to-end neural network weight compression approach tha...

Image Coding for Machines with Omnipotent Feature Learning

Image Coding for Machines (ICM) aims to compress images for AI tasks ana...

Learning to Localize Through Compressed Binary Maps

One of the main difficulties of scaling current localization systems to ...

DZip: improved general-purpose lossless compression based on novel neural network modeling

We consider lossless compression based on statistical data modeling foll...