Improve Convolutional Neural Network Pruning by Maximizing Filter Variety

03/11/2022
by   Nathan Hubens, et al.
9

Neural network pruning is a widely used strategy for reducing model storage and computing requirements. It allows to lower the complexity of the network by introducing sparsity in the weights. Because taking advantage of sparse matrices is still challenging, pruning is often performed in a structured way, i.e. removing entire convolution filters in the case of ConvNets, according to a chosen pruning criteria. Common pruning criteria, such as l1-norm or movement, usually do not consider the individual utility of filters, which may lead to: (1) the removal of filters exhibiting rare, thus important and discriminative behaviour, and (2) the retaining of filters with redundant information. In this paper, we present a technique solving those two issues, and which can be appended to any pruning criteria. This technique ensures that the criteria of selection focuses on redundant filters, while retaining the rare ones, thus maximizing the variety of remaining filters. The experimental results, carried out on different datasets (CIFAR-10, CIFAR-100 and CALTECH-101) and using different architectures (VGG-16 and ResNet-18) demonstrate that it is possible to achieve similar sparsity levels while maintaining a higher performance when appending our filter selection technique to pruning criteria. Moreover, we assess the quality of the found sparse sub-networks by applying the Lottery Ticket Hypothesis and find that the addition of our method allows to discover better performing tickets in most cases

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/16/2023

WHC: Weighted Hybrid Criterion for Filter Pruning on Convolutional Neural Networks

Filter pruning has attracted increasing attention in recent years for it...
research
07/03/2023

Structured Network Pruning by Measuring Filter-wise Interactions

Structured network pruning is a practical approach to reduce computation...
research
01/30/2021

Deep Model Compression based on the Training History

Deep Convolutional Neural Networks (DCNNs) have shown promising results ...
research
11/18/2018

RePr: Improved Training of Convolutional Filters

A well-trained Convolutional Neural Network can easily be pruned without...
research
06/22/2023

Neural Network Pruning for Real-time Polyp Segmentation

Computer-assisted treatment has emerged as a viable application of medic...
research
03/18/2020

MINT: Deep Network Compression via Mutual Information-based Neuron Trimming

Most approaches to deep neural network compression via pruning either ev...
research
06/22/2020

Slimming Neural Networks using Adaptive Connectivity Scores

There are two broad approaches to deep neural network (DNN) pruning: 1) ...

Please sign up or login with your details

Forgot password? Click here to reset