Searching for Network Width with Bilaterally Coupled Network

03/25/2022
by   Xiu Su, et al.
0

Searching for a more compact network width recently serves as an effective way of channel pruning for the deployment of convolutional neural networks (CNNs) under hardware constraints. To fulfill the searching, a one-shot supernet is usually leveraged to efficiently evaluate the performance  different network widths. However, current methods mainly follow a unilaterally augmented (UA) principle for the evaluation of each width, which induces the training unfairness of channels in supernet. In this paper, we introduce a new supernet called Bilaterally Coupled Network (BCNet) to address this issue. In BCNet, each channel is fairly trained and responsible for the same amount of network widths, thus each network width can be evaluated more accurately. Besides, we propose to reduce the redundant search space and present the BCNetV2 as the enhanced supernet to ensure rigorous training fairness over channels. Furthermore, we leverage a stochastic complementary strategy for training the BCNet, and propose a prior initial population sampling method to boost the performance of the evolutionary search. We also propose the first open-source width benchmark on macro structures named Channel-Bench-Macro for the better comparison of width search algorithms. Extensive experiments on benchmark CIFAR-10 and ImageNet datasets indicate that our method can achieve state-of-the-art or competing performance over other baseline methods. Moreover, our method turns out to further boost the performance of NAS models by refining their network widths. For example, with the same FLOPs budget, our obtained EfficientNet-B0 achieves 77.53% Top-1 accuracy on ImageNet dataset, surpassing the performance of original setting by 0.65%.

READ FULL TEXT

page 5

page 7

page 11

page 12

page 14

research
05/21/2021

BCNet: Searching for Network Width with Bilaterally Coupled Network

Searching for a more compact network width recently serves as an effecti...
research
02/10/2021

Locally Free Weight Sharing for Network Width Search

Searching for network width is an effective way to slim deep neural netw...
research
03/22/2021

Prioritized Architecture Sampling with Monto-Carlo Tree Search

One-shot neural architecture search (NAS) methods significantly reduce t...
research
05/23/2019

Network Pruning via Transformable Architecture Search

Network pruning reduces the computation costs of an over-parameterized n...
research
05/11/2022

Revisiting Random Channel Pruning for Neural Network Compression

Channel (or 3D filter) pruning serves as an effective way to accelerate ...
research
04/24/2021

Width Transfer: On the (In)variance of Width Optimization

Optimizing the channel counts for different layers of a CNN has shown gr...
research
03/30/2021

Differentiable Network Adaption with Elastic Search Space

In this paper we propose a novel network adaption method called Differen...

Please sign up or login with your details

Forgot password? Click here to reset