Softer Pruning, Incremental Regularization

10/19/2020
by   Linhang Cai, et al.
0

Network pruning is widely used to compress Deep Neural Networks (DNNs). The Soft Filter Pruning (SFP) method zeroizes the pruned filters during training while updating them in the next training epoch. Thus the trained information of the pruned filters is completely dropped. To utilize the trained pruned filters, we proposed a SofteR Filter Pruning (SRFP) method and its variant, Asymptotic SofteR Filter Pruning (ASRFP), simply decaying the pruned weights with a monotonic decreasing parameter. Our methods perform well across various networks, datasets and pruning rates, also transferable to weight pruning. On ILSVRC-2012, ASRFP prunes 40 and 0.68 incremental regularization of the pruned filters. Besides, We note that SRFP and ASRFP pursue better results while slowing down the speed of convergence.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/06/2020

GHFP: Gradually Hard Filter Pruning

Filter pruning is widely used to reduce the computation of deep learning...
research
05/28/2019

Online Filter Clustering and Pruning for Efficient Convnets

Pruning filters is an effective method for accelerating deep neural netw...
research
01/15/2020

Filter Grafting for Deep Neural Networks

This paper proposes a new learning paradigm called filter grafting, whic...
research
10/28/2020

Data Agnostic Filter Gating for Efficient Deep Networks

To deploy a well-trained CNN model on low-end computation edge devices, ...
research
03/30/2020

How Not to Give a FLOP: Combining Regularization and Pruning for Efficient Inference

The challenge of speeding up deep learning models during the deployment ...
research
10/22/2022

Sub-network Multi-objective Evolutionary Algorithm for Filter Pruning

Filter pruning is a common method to achieve model compression and accel...
research
10/09/2021

Weight Evolution: Improving Deep Neural Networks Training through Evolving Inferior Weight Values

To obtain good performance, convolutional neural networks are usually ov...

Please sign up or login with your details

Forgot password? Click here to reset