FBNetV3: Joint Architecture-Recipe Search using Neural Acquisition Function

06/03/2020
by   Xiaoliang Dai, et al.
11

Neural Architecture Search (NAS) yields state-of-the-art neural networks that outperform their best manually-designed counterparts. However, previous NAS methods search for architectures under one training recipe (i.e., training hyperparameters), ignoring the significance of training recipes and overlooking superior architectures under other training recipes. Thus, they fail to find higher-accuracy architecture-recipe combinations. To address this oversight, we present JointNAS to search both (a) architectures and (b) their corresponding training recipes. To accomplish this, we introduce a neural acquisition function that scores architectures and training recipes jointly. Following pre-training on a proxy dataset, this acquisition function guides both coarse-grained and fine-grained searches to produce FBNetV3. FBNetV3 is a family of state-of-the-art compact ImageNet models, outperforming both automatically and manually-designed architectures. For example, FBNetV3 matches both EfficientNet and ResNeSt accuracy with 1.4x and 5.0x fewer FLOPs, respectively. Furthermore, the JointNAS-searched training recipe yields significant performance gains across different networks and tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/17/2020

Fine-Grained Stochastic Architecture Search

State-of-the-art deep networks are often too large to deploy on mobile d...
research
01/22/2022

NAS-VAD: Neural Architecture Search for Voice Activity Detection

The need for automatic design of deep neural networks has led to the eme...
research
11/30/2020

Optimizing the Neural Architecture of Reinforcement Learning Agents

Reinforcement learning (RL) enjoyed significant progress over the last y...
research
12/03/2020

Automatic Routability Predictor Development Using Neural Architecture Search

The rise of machine learning technology inspires a boom of its applicati...
research
06/12/2020

Does Unsupervised Architecture Representation Learning Help Neural Architecture Search?

Existing Neural Architecture Search (NAS) methods either encode neural a...
research
07/15/2019

Batch-Shaped Channel Gated Networks

We present a method for gating deep-learning architectures on a fine-gra...
research
06/19/2019

SwiftNet: Using Graph Propagation as Meta-knowledge to Search Highly Representative Neural Architectures

Designing neural architectures for edge devices is subject to constraint...

Please sign up or login with your details

Forgot password? Click here to reset