LEAN: graph-based pruning for convolutional neural networks by extracting longest chains

11/13/2020
by   Richard Schoonhoven, et al.
0

Convolutional neural networks (CNNs) have proven to be highly successful at a range of image-to-image tasks. CNNs can be computationally expensive, which can limit their applicability in practice. Model pruning can improve computational efficiency by sparsifying trained networks. Common methods for pruning CNNs determine what convolutional filters to remove by ranking filters on an individual basis. However, filters are not independent, as CNNs consist of chains of convolutions, which can result in sub-optimal filter selection. We propose a novel pruning method, LongEst-chAiN (LEAN) pruning, which takes the interdependency between the convolution operations into account. We propose to prune CNNs by using graph-based algorithms to select relevant chains of convolutions. A CNN is interpreted as a graph, with the operator norm of each convolution as distance metric for the edges. LEAN pruning iteratively extracts the highest value path from the graph to keep. In our experiments, we test LEAN pruning for several image-to-image tasks, including the well-known CamVid dataset. LEAN pruning enables us to keep just 0.5 without significant loss of accuracy. When pruning CNNs with LEAN, we achieve a higher accuracy than pruning filters individually, and different pruned substructures emerge.

READ FULL TEXT

page 6

page 8

research
05/12/2019

Approximated Oracle Filter Pruning for Destructive CNN Width Optimization

It is not easy to design and run Convolutional Neural Networks (CNNs) du...
research
04/05/2023

Efficient CNNs via Passive Filter Pruning

Convolutional neural networks (CNNs) have shown state-of-the-art perform...
research
01/23/2018

Learning to Prune Filters in Convolutional Neural Networks

Many state-of-the-art computer vision algorithms use large scale convolu...
research
10/29/2018

Demystifying Neural Network Filter Pruning

Based on filter magnitude ranking (e.g. L1 norm), conventional filter pr...
research
04/08/2019

Centripetal SGD for Pruning Very Deep Convolutional Networks with Complicated Structure

The redundancy is widely recognized in Convolutional Neural Networks (CN...
research
06/03/2022

Pruning for Interpretable, Feature-Preserving Circuits in CNNs

Deep convolutional neural networks are a powerful model class for a rang...
research
06/11/2019

A Taxonomy of Channel Pruning Signals in CNNs

Convolutional neural networks (CNNs) are widely used for classification ...

Please sign up or login with your details

Forgot password? Click here to reset