Layer Pruning via Fusible Residual Convolutional Block for Deep Neural Networks

11/29/2020
by   Pengtao Xu, et al.
0

In order to deploy deep convolutional neural networks (CNNs) on resource-limited devices, many model pruning methods for filters and weights have been developed, while only a few to layer pruning. However, compared with filter pruning and weight pruning, the compact model obtained by layer pruning has less inference time and run-time memory usage when the same FLOPs and number of parameters are pruned because of less data moving in memory. In this paper, we propose a simple layer pruning method using fusible residual convolutional block (ResConv), which is implemented by inserting shortcut connection with a trainable information control parameter into a single convolutional layer. Using ResConv structures in training can improve network accuracy and train deep plain networks, and adds no additional computation during inference process because ResConv is fused to be an ordinary convolutional layer after training. For layer pruning, we convert convolutional layers of network into ResConv with a layer scaling factor. In the training process, the L1 regularization is adopted to make the scaling factors sparse, so that unimportant layers are automatically identified and then removed, resulting in a model of layer reduction. Our pruning method achieves excellent performance of compression and acceleration over the state-of-the-arts on different datasets, and needs no retraining in the case of low pruning rate. For example, with ResNet-110, we achieve a 65.5 55.5 CIFAR-10.

READ FULL TEXT
research
10/03/2020

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

To apply deep CNNs to mobile terminals and portable devices, many schola...
research
12/21/2019

DBP: Discrimination Based Block-Level Pruning for Deep Model Acceleration

Neural network pruning is one of the most popular methods of acceleratin...
research
01/13/2022

Recursive Least Squares for Training and Pruning Convolutional Neural Networks

Convolutional neural networks (CNNs) have succeeded in many practical ap...
research
09/23/2020

Pruning Convolutional Filters using Batch Bridgeout

State-of-the-art computer vision models are rapidly increasing in capaci...
research
12/17/2019

ℓ_0 Regularized Structured Sparsity Convolutional Neural Networks

Deepening and widening convolutional neural networks (CNNs) significantl...
research
08/26/2022

Complexity-Driven CNN Compression for Resource-constrained Edge AI

Recent advances in Artificial Intelligence (AI) on the Internet of Thing...
research
11/20/2020

Continuous Pruning of Deep Convolutional Networks Using Selective Weight Decay

During the last decade, deep convolutional networks have become the refe...

Please sign up or login with your details

Forgot password? Click here to reset