DeepAI AI Chat
Log In Sign Up

Evolving Search Space for Neural Architecture Search

11/22/2020
by   Yuanzheng Ci, et al.
0

The automation of neural architecture design has been a coveted alternative to human experts. Recent works have small search space, which is easier to optimize but has a limited upper bound of the optimal solution. Extra human design is needed for those methods to propose a more suitable space with respect to the specific task and algorithm capacity. To further enhance the degree of automation for neural architecture search, we present a Neural Search-space Evolution (NSE) scheme that iteratively amplifies the results from the previous effort by maintaining an optimized search space subset. This design minimizes the necessity of a well-designed search space. We further extend the flexibility of obtainable architectures by introducing a learnable multi-branch setting. By employing the proposed method, a consistent performance gain is achieved during a progressive search over upcoming search spaces. We achieve 77.3 which yielded a state-of-the-art performance among previous auto-generated architectures that do not involve knowledge distillation or weight pruning. When the latency constraint is adopted, our result also performs better than the previous best-performing mobile models with a 77.9

READ FULL TEXT
03/03/2022

Neural Architecture Search using Progressive Evolution

Vanilla neural architecture search using evolutionary algorithms (EA) in...
02/12/2022

Evolving Neural Networks with Optimal Balance between Information Flow and Connections Cost

Evolving Neural Networks (NNs) has recently seen an increasing interest ...
02/08/2021

Contrastive Embeddings for Neural Architectures

The performance of algorithms for neural architecture search strongly de...
03/23/2021

Neural Architecture Search From Fréchet Task Distance

We formulate a Fréchet-type asymmetric distance between tasks based on F...
07/31/2020

HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization

Neural Architecture Search has achieved state-of-the-art performance in ...
09/30/2019

Towards modular and programmable architecture search

Neural architecture search methods are able to find high performance dee...
03/02/2020

RandomNet: Towards Fully Automatic Neural Architecture Design for Multimodal Learning

Almost all neural architecture search methods are evaluated in terms of ...