Neural Architecture Search with Random Labels

01/28/2021
by   Xuanyang Zhang, et al.
0

In this paper, we investigate a new variant of neural architecture search (NAS) paradigm – searching with random labels (RLNAS). The task sounds counter-intuitive for most existing NAS algorithms since random label provides few information on the performance of each candidate architecture. Instead, we propose a novel NAS framework based on ease-of-convergence hypothesis, which requires only random labels during searching. The algorithm involves two steps: first, we train a SuperNet using random labels; second, from the SuperNet we extract the sub-network whose weights change most significantly during the training. Extensive experiments are evaluated on multiple datasets (e.g. NAS-Bench-201 and ImageNet) and multiple search spaces (e.g. DARTS-like and MobileNet-like). Very surprisingly, RLNAS achieves comparable or even better results compared with state-of-the-art NAS methods such as PC-DARTS, Single Path One-Shot, even though the counterparts utilize full ground truth labels for searching. We hope our finding could inspire new understandings on the essential of NAS.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/16/2021

BN-NAS: Neural Architecture Search with Batch Normalization

We present BN-NAS, neural architecture search with Batch Normalization (...
research
05/27/2020

Synthetic Petri Dish: A Novel Surrogate Model for Rapid Architecture Search

Neural Architecture Search (NAS) explores a large space of architectural...
research
01/11/2021

PEng4NN: An Accurate Performance Estimation Engine for Efficient Automated Neural Network Architecture Search

Neural network (NN) models are increasingly used in scientific simulatio...
research
12/03/2021

Data-Free Neural Architecture Search via Recursive Label Calibration

This paper aims to explore the feasibility of neural architecture search...
research
06/23/2019

One-Shot Neural Architecture Search Through A Posteriori Distribution Guided Sampling

The emergence of one-shot approaches has greatly advanced the research o...
research
08/18/2022

Differentiable Architecture Search with Random Features

Differentiable architecture search (DARTS) has significantly promoted th...
research
08/28/2020

NATS-Bench: Benchmarking NAS algorithms for Architecture Topology and Size

Neural architecture search (NAS) has attracted a lot of attention and ha...

Please sign up or login with your details

Forgot password? Click here to reset