BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels

03/22/2020
by   Zan Shen, et al.
0

One-Shot methods have evolved into one of the most popular methods in Neural Architecture Search (NAS) due to weight sharing and single training of a supernet. However, existing methods generally suffer from two issues: predetermined number of channels in each layer which is suboptimal; and model averaging effects and poor ranking correlation caused by weight coupling and continuously expanding search space. To explicitly address these issues, in this paper, a Broadening-and-Shrinking One-Shot NAS (BS-NAS) framework is proposed, in which `broadening' refers to broadening the search space with a spring block enabling search for numbers of channels during training of the supernet; while `shrinking' refers to a novel shrinking strategy gradually turning off those underperforming operations. The above innovations broaden the search space for wider representation and then shrink it by gradually removing underperforming operations, followed by an evolutionary algorithm to efficiently search for the optimal architecture. Extensive experiments on ImageNet illustrate the effectiveness of the proposed BS-NAS as well as the state-of-the-art performance.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/28/2020

Angle-based Search Space Shrinking for Neural Architecture Search

In this work, we present a simple and general search space shrinking met...
research
05/15/2020

HNAS: Hierarchical Neural Architecture Search on Mobile Devices

Neural Architecture Search (NAS) has attracted growing interest. To redu...
research
04/27/2022

PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Search

Neural architecture search (NAS) aims to automate architecture engineeri...
research
02/10/2021

Locally Free Weight Sharing for Network Width Search

Searching for network width is an effective way to slim deep neural netw...
research
10/06/2019

Improving One-shot NAS by Suppressing the Posterior Fading

There is a growing interest in automated neural architecture search (NAS...
research
08/01/2021

WAS-VTON: Warping Architecture Search for Virtual Try-on Network

Despite recent progress on image-based virtual try-on, current methods a...
research
07/16/2022

CLOSE: Curriculum Learning On the Sharing Extent Towards Better One-shot NAS

One-shot Neural Architecture Search (NAS) has been widely used to discov...

Please sign up or login with your details

Forgot password? Click here to reset