DeepAI AI Chat
Log In Sign Up

DeepCABAC: A Universal Compression Algorithm for Deep Neural Networks

07/27/2019
by   Simon Wiedemann, et al.
Fraunhofer
0

The field of video compression has developed some of the most sophisticated and efficient compression algorithms known in the literature, enabling very high compressibility for little loss of information. Whilst some of these techniques are domain specific, many of their underlying principles are universal in that they can be adapted and applied for compressing different types of data. In this work we present DeepCABAC, a compression algorithm for deep neural networks that is based on one of the state-of-the-art video coding techniques. Concretely, it applies a Context-based Adaptive Binary Arithmetic Coder (CABAC) to the network's parameters, which was originally designed for the H.264/AVC video coding standard and became the state-of-the-art for lossless compression. Moreover, DeepCABAC employs a novel quantization scheme that minimizes the rate-distortion function while simultaneously taking the impact of quantization onto the accuracy of the network into account. Experimental results show that DeepCABAC consistently attains higher compression rates than previously proposed coding techniques for neural network compression. For instance, it is able to compress the VGG16 ImageNet model by x63.6 with no loss of accuracy, thus being able to represent the entire network with merely 8.7MB. The source code for encoding and decoding can be found at https://github.com/fraunhoferhhi/DeepCABAC.

READ FULL TEXT
05/15/2019

DeepCABAC: Context-adaptive binary arithmetic coding for deep neural network compression

We present DeepCABAC, a novel context-adaptive binary arithmetic coder f...
02/07/2018

Universal Deep Neural Network Compression

Compression of deep neural networks (DNNs) for memory- and computation-e...
06/09/2020

Neural Network Activation Quantization with Bitwise Information Bottlenecks

Recent researches on information bottleneck shed new light on the contin...
10/25/2017

End-to-End Optimized Speech Coding with Deep Neural Networks

Modern compression algorithms are often the result of laborious domain-s...
12/11/2020

Parallelized Rate-Distortion Optimized Quantization Using Deep Learning

Rate-Distortion Optimized Quantization (RDOQ) has played an important ro...
05/07/2023

Learned Wyner-Ziv Compressors Recover Binning

We consider lossy compression of an information source when the decoder ...
11/19/2021

Instance-Adaptive Video Compression: Improving Neural Codecs by Training on the Test Set

We introduce a video compression algorithm based on instance-adaptive le...