XferNAS: Transfer Neural Architecture Search

07/18/2019
by   Martin Wistuba, et al.
0

The term Neural Architecture Search (NAS) refers to the automatic optimization of network architectures for a new, previously unknown task. Since testing an architecture is computationally very expensive, many optimizers need days or even weeks to find suitable architectures. However, this search time can be significantly reduced if knowledge from previous searches on different tasks is reused. In this work, we propose a generally applicable framework that introduces only minor changes to existing optimizers to leverage this feature. As an example, we select an existing optimizer and demonstrate the complexity of the integration of the framework as well as its impact. In experiments on CIFAR-10 and CIFAR-100, we observe a reduction in the search time from 200 to only 6 GPU days, a speed up by a factor of 33. In addition, we observe new records of 1.99 and 14.06 for NAS optimizers on the CIFAR benchmarks, respectively. In a separate study, we analyze the impact of the amount of source and target data. Empirically, we demonstrate that the proposed framework generally gives better results and, in the worst case, is just as good as the unmodified optimizer.

READ FULL TEXT
research
07/09/2023

Carbon-Efficient Neural Architecture Search

This work presents a novel approach to neural architecture search (NAS) ...
research
12/23/2020

Evolving Neural Architecture Using One Shot Model

Neural Architecture Search (NAS) is emerging as a new research direction...
research
05/13/2019

BayesNAS: A Bayesian Approach for Neural Architecture Search

One-Shot Neural Architecture Search (NAS) is a promising method to signi...
research
09/08/2020

Binarized Neural Architecture Search for Efficient Object Recognition

Traditional neural architecture search (NAS) has a significant impact in...
research
07/15/2021

Neural Architecture Search using Covariance Matrix Adaptation Evolution Strategy

Evolution-based neural architecture search requires high computational r...
research
03/08/2019

Inductive Transfer for Neural Architecture Optimization

The recent advent of automated neural network architecture search led to...
research
12/20/2017

Finding Competitive Network Architectures Within a Day Using UCT

The design of neural network architectures for a new data set is a labor...

Please sign up or login with your details

Forgot password? Click here to reset