Differential Evolution for Neural Architecture Search

by   Noor Awad, et al.

Neural architecture search (NAS) methods rely on a search strategy for deciding which architectures to evaluate next and a performance estimation strategy for assessing their performance (e.g., using full evaluations, multi-fidelity evaluations, or the one-shot model). In this paper, we focus on the search strategy. We introduce the simple yet powerful evolutionary algorithm of differential evolution to the NAS community. Using the simplest performance evaluation strategy of full evaluations, we comprehensively compare this search strategy to regularized evolution and Bayesian optimization and demonstrate that it yields improved and more robust results for 13 tabular NAS benchmarks based on NAS-Bench-101, NAS-Bench-1Shot1, NAS-Bench-201 and NAS-HPO bench.


page 1

page 2

page 3

page 4


NAS-Bench-1Shot1: Benchmarking and Dissecting One-shot Neural Architecture Search

One-shot neural architecture search (NAS) has played a crucial role in m...

BANANAS: Bayesian Optimization with Neural Architectures for Neural Architecture Search

Neural Architecture Search (NAS) has seen an explosion of research in th...

EPE-NAS: Efficient Performance Estimation Without Training for Neural Architecture Search

Neural Architecture Search (NAS) has shown excellent results in designin...

B2EA: An Evolutionary Algorithm Assisted by Two Bayesian Optimization Modules for Neural Architecture Search

The early pioneering Neural Architecture Search (NAS) works were multi-t...

Heed the Noise in Performance Evaluations in Neural Architecture Search

Neural Architecture Search (NAS) has recently become a topic of great in...

How to Simplify Search: Classification-wise Pareto Evolution for One-shot Neural Architecture Search

In the deployment of deep neural models, how to effectively and automati...

When NAS Meets Watermarking: Ownership Verification of DNN Models via Cache Side Channels

We present a novel watermarking scheme to verify the ownership of DNN mo...

Code Repositories