Data-free parameter pruning for Deep Neural Networks

07/22/2015
by   Suraj Srinivas, et al.
0

Deep Neural nets (NNs) with millions of parameters are at the heart of many state-of-the-art computer vision systems today. However, recent works have shown that much smaller models can achieve similar levels of performance. In this work, we address the problem of pruning parameters in a trained NN model. Instead of removing individual weights one at a time as done in previous works, we remove one neuron at a time. We show how similar neurons are redundant, and propose a systematic way to remove them. Our experiments in pruning the densely connected layers show that we can remove upto 85% of the total parameters in an MNIST-trained network, and about 35% for AlexNet without significantly affecting performance. Our method can be applied on top of most networks with a fully connected layer to give a smaller network.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/29/2017

Towards thinner convolutional neural networks through Gradually Global Pruning

Deep network pruning is an effective method to reduce the storage and co...
research
11/16/2021

Neuron-based Pruning of Deep Neural Networks with Better Generalization using Kronecker Factored Curvature Approximation

Existing methods of pruning deep neural networks focus on removing unnec...
research
11/17/2015

Learning Neural Network Architectures using Backpropagation

Deep neural networks with millions of parameters are at the heart of man...
research
01/15/2021

Convolutional Neural Network with Pruning Method for Handwritten Digit Recognition

CNN model is a popular method for imagery analysis, so it could be utili...
research
02/08/2022

EvoPruneDeepTL: An Evolutionary Pruning Model for Transfer Learning based Deep Neural Networks

In recent years, Deep Learning models have shown a great performance in ...
research
06/15/2018

Detecting Dead Weights and Units in Neural Networks

Deep Neural Networks are highly over-parameterized and the size of the n...
research
03/21/2018

Fisher Pruning of Deep Nets for Facial Trait Classification

Although deep nets have resulted in high accuracies for various visual t...

Please sign up or login with your details

Forgot password? Click here to reset