Finding Competitive Network Architectures Within a Day Using UCT

by   Martin Wistuba, et al.

The design of neural network architectures for a new data set is a laborious task which requires human deep learning expertise. In order to make deep learning available for a broader audience, automated methods for finding a neural network architecture are vital. Recently proposed methods can already achieve human expert level performances. However, these methods have run times of months or even years of GPU computing time, ignoring hardware constraints as faced by many researchers and companies. We propose the use of Monte Carlo planning in combination with two different UCT (upper confidence bound applied to trees) derivations to search for network architectures. We adapt the UCT algorithm to the needs of network architecture search by proposing two ways of sharing information between different branches of the search tree. In an empirical study we are able to demonstrate that this method is able to find competitive networks for MNIST, SVHN and CIFAR-10 in just a single GPU day. Extending the search time to five GPU days, we are able to outperform human architectures and our competitors which consider the same types of layers.



page 1

page 2

page 3

page 4


Resource Constrained Neural Network Architecture Search

The design of neural network architectures is frequently either based on...

Simple And Efficient Architecture Search for Convolutional Neural Networks

Neural networks have recently had a lot of success for many tasks. Howev...

EEEA-Net: An Early Exit Evolutionary Neural Architecture Search

The goals of this research were to search for Convolutional Neural Netwo...

Multi-objective Architecture Search for CNNs

Architecture search aims at automatically finding neural architectures t...

XferNAS: Transfer Neural Architecture Search

The term Neural Architecture Search (NAS) refers to the automatic optimi...

Exascale Deep Learning to Accelerate Cancer Research

Deep learning, through the use of neural networks, has demonstrated rema...

Architecture Search by Estimation of Network Structure Distributions

The influence of deep learning is continuously expanding across differen...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.