AACP: Model Compression by Accurate and Automatic Channel Pruning

01/31/2021
by   Lanbo Lin, et al.
0

Channel pruning is formulated as a neural architecture search (NAS) problem recently. However, existing NAS-based methods are challenged by huge computational cost and inflexibility of applications. How to deal with multiple sparsity constraints simultaneously and speed up NAS-based channel pruning are still open challenges. In this paper, we propose a novel Accurate and Automatic Channel Pruning (AACP) method to address these problems. Firstly, AACP represents the structure of a model as a structure vector and introduces a pruning step vector to control the compressing granularity of each layer. Secondly, AACP utilizes Pruned Structure Accuracy Estimator (PSAE) to speed up the performance estimation process. Thirdly, AACP proposes Improved Differential Evolution (IDE) algorithm to search the optimal structure vector effectively. Because of IDE, AACP can deal with FLOPs constraint and model size constraint simultaneously and efficiently. Our method can be easily applied to various tasks and achieve state of the art performance. On CIFAR10, our method reduces 65% FLOPs of ResNet110 with an improvement of 0.26% top-1 accuracy. On ImageNet, we reduce 42% FLOPs of ResNet50 with a small loss of 0.18% top-1 accuracy and reduce 30% FLOPs of MobileNetV2 with a small loss of 0.7% top-1 accuracy. The source code will be released after publication.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/02/2022

Pruning-as-Search: Efficient Neural Architecture Search via Channel Pruning and Structural Reparameterization

Neural architecture search (NAS) and network pruning are widely studied ...
research
03/08/2022

UENAS: A Unified Evolution-based NAS Framework

Neural architecture search (NAS) has gained significant attention for au...
research
03/17/2023

Dynamic Structure Pruning for Compressing CNNs

Structure pruning is an effective method to compress and accelerate neur...
research
12/31/2021

Multi-Dimensional Model Compression of Vision Transformer

Vision transformers (ViT) have recently attracted considerable attention...
research
11/12/2019

CALPA-NET: Channel-pruning-assisted Deep Residual Network for Steganalysis of Digital Images

Over the past few years, detection performance improvements of deep-lear...
research
02/19/2020

Knapsack Pruning with Inner Distillation

Neural network pruning reduces the computational cost of an over-paramet...
research
08/19/2021

An Information Theory-inspired Strategy for Automatic Network Pruning

Despite superior performance on many computer vision tasks, deep convolu...

Please sign up or login with your details

Forgot password? Click here to reset