DeepAI
Log In Sign Up

Unchain the Search Space with Hierarchical Differentiable Architecture Search

01/11/2021
by   Guanting Liu, et al.
0

Differentiable architecture search (DAS) has made great progress in searching for high-performance architectures with reduced computational cost. However, DAS-based methods mainly focus on searching for a repeatable cell structure, which is then stacked sequentially in multiple stages to form the networks. This configuration significantly reduces the search space, and ignores the importance of connections between the cells. To overcome this limitation, in this paper, we propose a Hierarchical Differentiable Architecture Search (H-DAS) that performs architecture search both at the cell level and at the stage level. Specifically, the cell-level search space is relaxed so that the networks can learn stage-specific cell structures. For the stage-level search, we systematically study the architectures of stages, including the number of cells in each stage and the connections between the cells. Based on insightful observations, we design several search rules and losses, and mange to search for better stage-level architectures. Such hierarchical search space greatly improves the performance of the networks without introducing expensive search cost. Extensive experiments on CIFAR10 and ImageNet demonstrate the effectiveness of the proposed H-DAS. Moreover, the searched stage-level architectures can be combined with the cell structures searched by existing DAS methods to further boost the performance. Code is available at: https://github.com/MalongTech/research-HDAS

READ FULL TEXT

page 1

page 2

page 3

page 4

05/09/2021

Lightweight Image Super-Resolution with Hierarchical and Differentiable Neural Architecture Search

Single Image Super-Resolution (SISR) tasks have achieved significant per...
07/06/2020

Multi-Objective Neural Architecture Search Based on Diverse Structures and Adaptive Recommendation

The search space of neural architecture search (NAS) for convolutional n...
10/02/2020

DOTS: Decoupling Operation and Topology in Differentiable Architecture Search

Differentiable Architecture Search (DARTS) has attracted extensive atten...
06/16/2020

AlphaGAN: Fully Differentiable Architecture Search for Generative Adversarial Networks

Generative Adversarial Networks (GANs) are formulated as minimax game pr...
08/26/2019

Customizable Architecture Search for Semantic Segmentation

In this paper, we propose a Customizable Architecture Search (CAS) appro...
06/13/2022

Differentiable and Transportable Structure Learning

We are interested in unsupervised structure learning with a particular f...
09/15/2019

Graph-guided Architecture Search for Real-time Semantic Segmentation

Designing a lightweight semantic segmentation network often requires res...