DeepAI AI Chat
Log In Sign Up

Evolving Neural Architecture Using One Shot Model

by   Nilotpal Sinha, et al.

Neural Architecture Search (NAS) is emerging as a new research direction which has the potential to replace the hand-crafted neural architectures designed for specific tasks. Previous evolution based architecture search requires high computational resources resulting in high search time. In this work, we propose a novel way of applying a simple genetic algorithm to the NAS problem called EvNAS (Evolving Neural Architecture using One Shot Model) which reduces the search time significantly while still achieving better result than previous evolution based methods. The architectures are represented by using the architecture parameter of the one shot model which results in the weight sharing among the architectures for a given population of architectures and also weight inheritance from one generation to the next generation of architectures. We propose a decoding technique for the architecture parameter which is used to divert majority of the gradient information towards the given architecture and is also used for improving the performance prediction of the given architecture from the one shot model during the search process. Furthermore, we use the accuracy of the partially trained architecture on the validation data as a prediction of its fitness in order to reduce the search time. EvNAS searches for the architecture on the proxy dataset i.e. CIFAR-10 for 4.4 GPU day on a single GPU and achieves top-1 test error of 2.47 3.63M parameters which is then transferred to CIFAR-100 and ImageNet achieving top-1 error of 16.37 results show the potential of evolutionary methods in solving the architecture search problem.


page 1

page 2

page 3

page 4


Neural Architecture Search using Covariance Matrix Adaptation Evolution Strategy

Evolution-based neural architecture search requires high computational r...

BayesNAS: A Bayesian Approach for Neural Architecture Search

One-Shot Neural Architecture Search (NAS) is a promising method to signi...

Efficient Novelty-Driven Neural Architecture Search

One-Shot Neural architecture search (NAS) attracts broad attention recen...

Novelty Driven Evolutionary Neural Architecture Search

Evolutionary algorithms (EA) based neural architecture search (NAS) invo...

XferNAS: Transfer Neural Architecture Search

The term Neural Architecture Search (NAS) refers to the automatic optimi...

Evolution under Length Constraints for CNN Architecture design

In recent years, the CNN architectures designed by evolution algorithms ...

Neural Predictor for Neural Architecture Search

Neural Architecture Search methods are effective but often use complex a...