UNAS: Differentiable Architecture Search Meets Reinforcement Learning

12/16/2019
by   Arash Vahdat, et al.
20

Neural architecture search (NAS) aims to discover network architectures with desired properties such as high accuracy or low latency. Recently, differentiable NAS (DNAS) has demonstrated promising results while maintaining a search cost orders of magnitude lower than reinforcement learning (RL) based NAS. However, DNAS models can only optimize differentiable loss functions in search, and they require an accurate differentiable approximation of non-differentiable criteria. In this work, we present UNAS, a unified framework for NAS, that encapsulates recent DNAS and RL-based approaches under one framework. Our framework brings the best of both worlds, and it enables us to search for architectures with both differentiable and non-differentiable criteria in one unified framework while maintaining a low search cost. Further, we introduce a new objective function for search based on the generalization gap that prevents the selection of architectures prone to overfitting. We present extensive experiments on the CIFAR-10, CIFAR-100 and ImageNet datasets and we perform search in two fundamentally different search spaces. We show that UNAS obtains the state-of-the-art average accuracy on all three datasets when compared to the architectures searched in the DARTS space. Moreover, we show that UNAS can find an efficient and accurate architecture in the ProxylessNAS search space, that outperforms existing MobileNetV2 based architectures.

READ FULL TEXT
research
12/24/2018

SNAS: Stochastic Neural Architecture Search

We propose Stochastic Neural Architecture Search (SNAS), an economical e...
research
06/24/2018

DARTS: Differentiable Architecture Search

This paper addresses the scalability challenge of architecture search by...
research
06/12/2021

Zero-Cost Proxies Meet Differentiable Architecture Search

Differentiable neural architecture search (NAS) has attracted significan...
research
09/13/2021

RADARS: Memory Efficient Reinforcement Learning Aided Differentiable Neural Architecture Search

Differentiable neural architecture search (DNAS) is known for its capaci...
research
10/25/2019

Stabilizing DARTS with Amended Gradient Estimation on Architectural Parameters

Differentiable neural architecture search has been a popular methodology...
research
11/30/2021

Improving Differentiable Architecture Search with a Generative Model

In differentiable neural architecture search (NAS) algorithms like DARTS...
research
10/10/2022

LidarNAS: Unifying and Searching Neural Architectures for 3D Point Clouds

Developing neural models that accurately understand objects in 3D point ...

Please sign up or login with your details

Forgot password? Click here to reset