DeepAI AI Chat
Log In Sign Up

Improve Convolutional Neural Network Pruning by Maximizing Filter Variety

by   Nathan Hubens, et al.
University of Mons

Neural network pruning is a widely used strategy for reducing model storage and computing requirements. It allows to lower the complexity of the network by introducing sparsity in the weights. Because taking advantage of sparse matrices is still challenging, pruning is often performed in a structured way, i.e. removing entire convolution filters in the case of ConvNets, according to a chosen pruning criteria. Common pruning criteria, such as l1-norm or movement, usually do not consider the individual utility of filters, which may lead to: (1) the removal of filters exhibiting rare, thus important and discriminative behaviour, and (2) the retaining of filters with redundant information. In this paper, we present a technique solving those two issues, and which can be appended to any pruning criteria. This technique ensures that the criteria of selection focuses on redundant filters, while retaining the rare ones, thus maximizing the variety of remaining filters. The experimental results, carried out on different datasets (CIFAR-10, CIFAR-100 and CALTECH-101) and using different architectures (VGG-16 and ResNet-18) demonstrate that it is possible to achieve similar sparsity levels while maintaining a higher performance when appending our filter selection technique to pruning criteria. Moreover, we assess the quality of the found sparse sub-networks by applying the Lottery Ticket Hypothesis and find that the addition of our method allows to discover better performing tickets in most cases


page 1

page 2

page 3

page 4


WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks

Filter pruning has attracted increasing attention in recent years for it...

Deep Model Compression based on the Training History

Deep Convolutional Neural Networks (DCNNs) have shown promising results ...

Pruning Filter via Geometric Median for Deep Convolutional Neural Networks Acceleration

Previous works utilized "smaller-norm-less-important" criterion to prune...

REPrune: Filter Pruning via Representative Election

Even though norm-based filter pruning methods are widely accepted, it is...

RePr: Improved Training of Convolutional Filters

A well-trained Convolutional Neural Network can easily be pruned without...

Slimming Neural Networks using Adaptive Connectivity Scores

There are two broad approaches to deep neural network (DNN) pruning: 1) ...

CPOT: Channel Pruning via Optimal Transport

Recent advances in deep neural networks (DNNs) lead to tremendously grow...