Progressive Differentiable Architecture Search: Bridging the Depth Gap between Search and Evaluation

04/29/2019
by   Xin Chen, et al.
0

Recently, differentiable search methods have made major progress in reducing the computational costs of neural architecture search. However, these approaches often report lower accuracy in evaluating the searched architecture or transferring it to another dataset. This is arguably due to the large gap between the architecture depths in search and evaluation scenarios. In this paper, we present an efficient algorithm which allows the depth of searched architectures to grow gradually during the training procedure. This brings two issues, namely, heavier computational overheads and weaker search stability, which we solve using search space approximation and regularization, respectively. With a significantly reduced search time ( 7 hours on a single GPU), our approach achieves state-of-the-art performance on both the proxy dataset (CIFAR10 or CIFAR100) and the target dataset (ImageNet). Code is available at https://github.com/chenxin061/pdarts.

READ FULL TEXT
research
12/23/2019

Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

With the rapid development of neural architecture search (NAS), research...
research
08/18/2021

Single-DARTS: Towards Stable Architecture Search

Differentiable architecture search (DARTS) marks a milestone in Neural A...
research
04/12/2020

FBNetV2: Differentiable Neural Architecture Search for Spatial and Channel Dimensions

Differentiable Neural Architecture Search (DNAS) has demonstrated great ...
research
08/04/2019

MoGA: Searching Beyond MobileNetV3

The evolution of MobileNets has laid a solid foundation for neural netwo...
research
02/24/2020

Semi-Supervised Neural Architecture Search

Neural architecture search (NAS) relies on a good controller to generate...
research
07/12/2019

PC-DARTS: Partial Channel Connections for Memory-Efficient Differentiable Architecture Search

Differentiable architecture search (DARTS) provided a fast solution in f...
research
07/17/2020

Standing on the Shoulders of Giants: Hardware and Neural Architecture Co-Search with Hot Start

Hardware and neural architecture co-search that automatically generates ...

Please sign up or login with your details

Forgot password? Click here to reset