Accelerating Neural Architecture Search via Proxy Data

by   Byunggook Na, et al.

Despite the increasing interest in neural architecture search (NAS), the significant computational cost of NAS is a hindrance to researchers. Hence, we propose to reduce the cost of NAS using proxy data, i.e., a representative subset of the target data, without sacrificing search performance. Even though data selection has been used across various fields, our evaluation of existing selection methods for NAS algorithms offered by NAS-Bench-1shot1 reveals that they are not always appropriate for NAS and a new selection method is necessary. By analyzing proxy data constructed using various selection methods through data entropy, we propose a novel proxy data selection method tailored for NAS. To empirically demonstrate the effectiveness, we conduct thorough experiments across diverse datasets, search spaces, and NAS algorithms. Consequently, NAS algorithms with the proposed selection discover architectures that are competitive with those obtained using the entire dataset. It significantly reduces the search cost: executing DARTS with the proposed selection requires only 40 minutes on CIFAR-10 and 7.5 hours on ImageNet with a single GPU. Additionally, when the architecture searched on ImageNet using the proposed selection is inversely transferred to CIFAR-10, a state-of-the-art test error of 2.4% is yielded. Our code is available at


page 3

page 5

page 10

page 11

page 12

page 16


GeNAS: Neural Architecture Search with Better Generalization

Neural Architecture Search (NAS) aims to automatically excavate the opti...

Zero-Cost Proxies for Lightweight NAS

Neural Architecture Search (NAS) is quickly becoming the standard method...

Speeding up NAS with Adaptive Subset Selection

A majority of recent developments in neural architecture search (NAS) ha...

Less is More: Proxy Datasets in NAS approaches

Neural Architecture Search (NAS) defines the design of Neural Networks a...

Accelerating Evolutionary Neural Architecture Search via Multi-Fidelity Evaluation

Evolutionary neural architecture search (ENAS) has recently received inc...

Progressive DARTS: Bridging the Optimization Gap for NAS in the Wild

With the rapid development of neural architecture search (NAS), research...

DA-NAS: Data Adapted Pruning for Efficient Neural Architecture Search

Efficient search is a core issue in Neural Architecture Search (NAS). It...

Please sign up or login with your details

Forgot password? Click here to reset