Filter Sketch for Network Pruning

01/23/2020
by   Mingbao Lin, et al.
0

In this paper, we propose a novel network pruning approach by information preserving of pre-trained network weights (filters). Our approach, referred to as FilterSketch, encodes the second-order information of pre-trained weights, through which the model performance is recovered by fine-tuning the pruned network in an end-to-end manner. Network pruning with information preserving can be approximated as a matrix sketch problem, which is efficiently solved by the off-the-shelf Frequent Direction method. FilterSketch thereby requires neither training from scratch nor data-driven iterative optimization, leading to a magnitude-order reduction of time consumption in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3 and prunes 59.9 for ResNet-110. On ILSVRC-2012, it achieves a reduction of 45.5 removes 43.0 ResNet-50. Source codes of the proposed FilterSketch can be available at https://github.com/lmbxmu/FilterSketch.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/23/2020

Channel Pruning via Automatic Structure Search

Channel pruning is among the predominant approaches to compress deep neu...
research
07/01/2023

Filter Pruning for Efficient CNNs via Knowledge-driven Differential Filter Sampler

Filter pruning simultaneously accelerates the computation and reduces th...
research
01/20/2021

Non-Parametric Adaptive Network Pruning

Popular network pruning algorithms reduce redundant information by optim...
research
02/24/2020

HRank: Filter Pruning using High-Rank Feature Map

Neural network pruning offers a promising prospect to facilitate deployi...
research
09/24/2020

A Gradient Flow Framework For Analyzing Network Pruning

Recent network pruning methods focus on pruning models early-on in train...
research
06/18/2023

Instant Soup: Cheap Pruning Ensembles in A Single Pass Can Draw Lottery Tickets from Large Models

Large pre-trained transformers have been receiving explosive attention i...
research
02/25/2023

A Unified Framework for Soft Threshold Pruning

Soft threshold pruning is among the cutting-edge pruning methods with st...

Please sign up or login with your details

Forgot password? Click here to reset