Efficient Progressive Neural Architecture Search

08/01/2018
by   Juan-Manuel Pérez-Rúa, et al.
0

This paper addresses the difficult problem of finding an optimal neural architecture design for a given image classification task. We propose a method that aggregates two main results of the previous state-of-the-art in neural architecture search. These are, appealing to the strong sampling efficiency of a search scheme based on sequential model-based optimization (SMBO), and increasing training efficiency by sharing weights among sampled architectures. Sequential search has previously demonstrated its capabilities to find state-of-the-art neural architectures for image classification. However, its computational cost remains high, even unreachable under modest computational settings. Affording SMBO with weight-sharing alleviates this problem. On the other hand, progressive search with SMBO is inherently greedy, as it leverages a learned surrogate function to predict the validation error of neural architectures. This prediction is directly used to rank the sampled neural architectures. We propose to attenuate the greediness of the original SMBO method by relaxing the role of the surrogate function so it predicts architecture sampling probability instead. We demonstrate with experiments on the CIFAR-10 dataset that our method, denominated Efficient progressive neural architecture search (EPNAS), leads to increased search efficiency, while retaining competitiveness of found architectures.

READ FULL TEXT

page 4

page 7

research
12/02/2017

Progressive Neural Architecture Search

We propose a method for learning CNN structures that is more efficient t...
research
06/19/2019

XNAS: Neural Architecture Search with Expert Advice

This paper introduces a novel optimization method for differential neura...
research
08/12/2021

DARTS for Inverse Problems: a Study on Hyperparameter Sensitivity

Differentiable architecture search (DARTS) is a widely researched tool f...
research
02/15/2019

Fast Task-Aware Architecture Inference

Neural architecture search has been shown to hold great promise towards ...
research
03/25/2021

EfficientTDNN: Efficient Architecture Search for Speaker Recognition in the Wild

Speaker recognition refers to audio biometrics that utilizes acoustic ch...
research
10/25/2018

Fast Neural Architecture Search of Compact Semantic Segmentation Models via Auxiliary Cells

Automated design of architectures tailored for a specific task at hand i...
research
08/16/2023

ResBuilder: Automated Learning of Depth with Residual Structures

In this work, we develop a neural architecture search algorithm, termed ...

Please sign up or login with your details

Forgot password? Click here to reset