DECORE: Deep Compression with Reinforcement Learning

by   Manoj Alwani, et al.

Deep learning has become an increasingly popular and powerful option for modern pattern recognition systems. However, many deep neural networks have millions to billions of parameters, making them untenable for real-world applications with constraints on memory or latency. As a result, powerful network compression techniques are a must for the widespread adoption of deep learning. We present DECORE, a reinforcement learning approach to automate the network compression process. Using a simple policy gradient method to learn which neurons or channels to keep or remove, we are able to achieve compression rates 3x to 5x greater than contemporary approaches. In contrast with other architecture search methods, DECORE is simple and quick to train, requiring only a few hours of training on 1 GPU. When applied to standard network architectures on different datasets, our approach achieves 11x to 103x compression on different architectures while maintaining accuracies similar to those of the original, large networks.



There are no comments yet.


page 13


N2N Learning: Network to Network Compression via Policy Gradient Reinforcement Learning

While bigger and deeper neural network architectures continue to advance...

Compression strategies and space-conscious representations for deep neural networks

Recent advances in deep learning have made available large, powerful con...

Optimizing Memory Placement using Evolutionary Graph Reinforcement Learning

As modern neural networks have grown to billions of parameters, meeting ...

Taxonomy and Evaluation of Structured Compression of Convolutional Neural Networks

The success of deep neural networks in many real-world applications is l...

Exploiting Non-Linear Redundancy for Neural Model Compression

Deploying deep learning models, comprising of non-linear combination of ...

Finding Competitive Network Architectures Within a Day Using UCT

The design of neural network architectures for a new data set is a labor...

Towards Robust Data Hiding Against (JPEG) Compression: A Pseudo-Differentiable Deep Learning Approach

Data hiding is one widely used approach for protecting authentication an...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.