ScarletNAS: Bridging the Gap Between Scalability and Fairness in Neural Architecture Search

08/16/2019
by   Xiangxiang Chu, et al.
0

One-shot neural architecture search features fast training of a supernet in a single run. A pivotal issue for this weight-sharing approach is the lacking of scalability. A simple adjustment with identity block renders a scalable supernet but it arouses unstable training, which makes the subsequent model ranking unreliable. In this paper, we introduce linearly equivalent transformation to soothe training turbulence, providing with the proof that such transformed path is identical with the original one as per representational power. The overall method is named as SCARLET (SCAlable supeRnet with Linearly Equivalent Transformation). We show through experiments that linearly equivalent transformations can indeed harmonize the supernet training. With an EfficientNet-like search space and a multi-objective reinforced evolutionary backend, it generates a series of competitive models: Scarlet-A achieves 76.9 EfficientNet-B0 by a large margin; the shallower Scarlet-B exemplifies the proposed scalability which attains the same accuracy 76.3 with much fewer FLOPs; Scarlet-C scores competitive 75.6 sizes. The models and evaluation code are released online https://github.com/xiaomi-automl/ScarletNAS .

READ FULL TEXT

page 7

page 8

research
07/03/2019

FairNAS: Rethinking Evaluation Fairness of Weight Sharing Neural Architecture Search

The ability to rank models by its real strength is the key to Neural Arc...
research
09/24/2019

dAUTOMAP: decomposing AUTOMAP to achieve scalability and enhance performance

AUTOMAP is a promising generalized reconstruction approach, however, it ...
research
01/16/2020

MixPath: A Unified Approach for One-shot Neural Architecture Search

The expressiveness of search space is a key concern in neural architectu...
research
10/29/2020

Cream of the Crop: Distilling Prioritized Paths For One-Shot Neural Architecture Search

One-shot weight sharing methods have recently drawn great attention in n...
research
01/26/2021

Efficient Multi-objective Evolutionary 3D Neural Architecture Search for COVID-19 Detection with Chest CT Scans

COVID-19 pandemic has spread globally for months. Due to its long incuba...
research
11/05/2019

An improved transformation between Fibonacci FSRs and Galois FSRs

Feedback shift registers (FSRs), which have two configurations: Fibonacc...

Please sign up or login with your details

Forgot password? Click here to reset