Multi-objective Architecture Search for CNNs

by   Thomas Elsken, et al.

Architecture search aims at automatically finding neural architectures that are competitive with architectures designed by human experts. While recent approaches have come close to matching the predictive performance of manually designed architectures for image recognition, these approaches are problematic under constrained resources for two reasons: first, the architecture search itself requires vast computational resources for most proposed methods. Secondly, the found neural architectures are solely optimized for high predictive performance without penalizing excessive resource consumption. We address the first shortcoming by proposing NASH, an architecture search which considerable reduces the computational resources required for training novel architectures by applying network morphisms and aggressive learning rate schedules. On CIFAR10, NASH finds architectures with errors below 4 days. We address the second shortcoming by proposing Pareto-NASH, a method for multi-objective architecture search that allows approximating the Pareto-front of architectures under multiple objective, such as predictive performance and number of parameters, in a single run of the method. Within 56 GPU days of architecture search, Pareto-NASH finds a model with 4M parameters and test error of 3.5 of 4.6


page 1

page 2

page 3

page 4


DVOLVER: Efficient Pareto-Optimal Neural Network Architecture Search

Automatic search of neural network architectures is a standing research ...

Efficient Search of Multiple Neural Architectures with Different Complexities via Importance Sampling

Neural architecture search (NAS) aims to automate architecture design pr...

Multi-Criterion Evolutionary Design of Deep Convolutional Neural Networks

Convolutional neural networks (CNNs) are the backbones of deep learning ...

Multi-objective Neural Architecture Search via Non-stationary Policy Gradient

Multi-objective Neural Architecture Search (NAS) aims to discover novel ...

A greedy constructive algorithm for the optimization of neural network architectures

In this work we propose a new method to optimize the architecture of an ...

Finding Competitive Network Architectures Within a Day Using UCT

The design of neural network architectures for a new data set is a labor...

Deep Demosaicing for Edge Implementation

Most digital cameras use sensors coated with a Color Filter Array (CFA) ...