Regularized Evolution for Image Classifier Architecture Search

02/05/2018
by   Esteban Real, et al.
0

The effort devoted to hand-crafting image classifiers has motivated the use of architecture search to discover them automatically. Reinforcement learning and evolution have both shown promise for this purpose. This study introduces a regularized version of a popular asynchronous evolutionary algorithm. We rigorously compare it to the non-regularized form and to a highly-successful reinforcement learning baseline. Using the same hardware, compute effort and neural network training code, we conduct repeated experiments side-by-side, exploring different datasets, search spaces and scales. We show regularized evolution consistently produces models with similar or higher accuracy, across a variety of contexts without need for re-tuning parameters. In addition, regularized evolution exhibits considerably better performance than reinforcement learning at early search stages, suggesting it may be the better choice when fewer compute resources are available. This constitutes the first controlled comparison of the two search algorithms in this context. Finally, we present new architectures discovered with regularized evolution that we nickname AmoebaNets. These models set a new state of the art for CIFAR-10 (mean test error = 2.13 parameters), and reach the current state of the art for ImageNet (top-5 accuracy = 96.2

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/01/2017

Hierarchical Representations for Efficient Architecture Search

We explore efficient neural architecture search methods and present a si...
research
06/01/2018

TAPAS: Train-less Accuracy Predictor for Architecture Search

In recent years an increasing number of researchers and practitioners ha...
research
04/08/2019

Resource Constrained Neural Network Architecture Search

The design of neural network architectures is frequently either based on...
research
12/13/2018

IRLAS: Inverse Reinforcement Learning for Architecture Search

In this paper, we propose an inverse reinforcement learning method for a...
research
07/17/2020

Off-Policy Reinforcement Learning for Efficient and Effective GAN Architecture Search

In this paper, we introduce a new reinforcement learning (RL) based neur...
research
04/20/2022

SpiderNet: Hybrid Differentiable-Evolutionary Architecture Search via Train-Free Metrics

Neural Architecture Search (NAS) algorithms are intended to remove the b...
research
03/03/2017

Large-Scale Evolution of Image Classifiers

Neural networks have proven effective at solving difficult problems but ...

Please sign up or login with your details

Forgot password? Click here to reset