Structured Probabilistic Pruning for Convolutional Neural Network Acceleration

09/20/2017
by   Huan Wang, et al.
0

Although deep Convolutional Neural Network (CNN) has shown better performance in various computer vision tasks, its application is restricted by a significant increase in storage and computation. Among CNN simplification techniques, parameter pruning is a promising approach which aims at reducing the number of weights of various layers without intensively reducing the original accuracy. In this paper, we propose a novel progressive parameter pruning method, named Structured Probabilistic Pruning (SPP), which effectively prunes weights of convolutional layers in a probabilistic manner. Specifically, unlike existing deterministic pruning approaches, where unimportant weights are permanently eliminated, SPP introduces a pruning probability for each weight, and pruning is guided by sampling from the pruning probabilities. A mechanism is designed to increase and decrease pruning probabilities based on importance criteria for the training process. Experiments show that, with 4x speedup, SPP can accelerate AlexNet with only 0.3 0.8 directly applied to accelerate multi-branch CNN networks, such as ResNet, without specific adaptations. Our 2x speedup ResNet-50 only suffers 0.8 of top-5 accuracy on ImageNet. We further prove the effectiveness of our method on transfer learning task on Flower-102 dataset with AlexNet.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/25/2018

Structured Deep Neural Network Pruning by Varying Regularization Parameters

Convolutional Neural Networks (CNN's) are restricted by their massive co...
research
12/18/2019

Pruning by Explaining: A Novel Criterion for Deep Neural Network Pruning

The success of convolutional neural networks (CNNs) in various applicati...
research
07/07/2020

Lossless CNN Channel Pruning via Gradient Resetting and Convolutional Re-parameterization

Channel pruning (a.k.a. filter pruning) aims to slim down a convolutiona...
research
02/12/2020

Retrain or not retrain? – efficient pruning methods of deep CNN networks

Convolutional neural networks (CNN) play a major role in image processin...
research
10/01/2022

EAPruning: Evolutionary Pruning for Vision Transformers and CNNs

Structured pruning greatly eases the deployment of large neural networks...
research
01/12/2021

Convolutional Neural Network Simplification with Progressive Retraining

Kernel pruning methods have been proposed to speed up, simplify, and imp...
research
11/14/2022

Pruning Very Deep Neural Network Channels for Efficient Inference

In this paper, we introduce a new channel pruning method to accelerate v...

Please sign up or login with your details

Forgot password? Click here to reset