Evolving Search Space for Neural Architecture Search

11/22/2020
by   Yuanzheng Ci, et al.
0

The automation of neural architecture design has been a coveted alternative to human experts. Recent works have small search space, which is easier to optimize but has a limited upper bound of the optimal solution. Extra human design is needed for those methods to propose a more suitable space with respect to the specific task and algorithm capacity. To further enhance the degree of automation for neural architecture search, we present a Neural Search-space Evolution (NSE) scheme that iteratively amplifies the results from the previous effort by maintaining an optimized search space subset. This design minimizes the necessity of a well-designed search space. We further extend the flexibility of obtainable architectures by introducing a learnable multi-branch setting. By employing the proposed method, a consistent performance gain is achieved during a progressive search over upcoming search spaces. We achieve 77.3 which yielded a state-of-the-art performance among previous auto-generated architectures that do not involve knowledge distillation or weight pruning. When the latency constraint is adopted, our result also performs better than the previous best-performing mobile models with a 77.9

READ FULL TEXT
research
03/03/2022

Neural Architecture Search using Progressive Evolution

Vanilla neural architecture search using evolutionary algorithms (EA) in...
research
06/06/2023

ColdNAS: Search to Modulate for User Cold-Start Recommendation

Making personalized recommendation for cold-start users, who only have a...
research
02/12/2022

Evolving Neural Networks with Optimal Balance between Information Flow and Connections Cost

Evolving Neural Networks (NNs) has recently seen an increasing interest ...
research
02/08/2021

Contrastive Embeddings for Neural Architectures

The performance of algorithms for neural architecture search strongly de...
research
07/31/2020

HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization

Neural Architecture Search has achieved state-of-the-art performance in ...
research
09/30/2019

Towards modular and programmable architecture search

Neural architecture search methods are able to find high performance dee...
research
01/16/2018

GitGraph - Architecture Search Space Creation through Frequent Computational Subgraph Mining

The dramatic success of deep neural networks across multiple application...

Please sign up or login with your details

Forgot password? Click here to reset