Angle-based Search Space Shrinking for Neural Architecture Search

04/28/2020
by   Yiming Hu, et al.
0

In this work, we present a simple and general search space shrinking method, called Angle-Based search space Shrinking (ABS), for Neural Architecture Search (NAS). Our approach progressively simplifies the original search space by dropping unpromising candidates, thus can reduce difficulties for existing NAS methods to find superior architectures. In particular, we propose an angle-based metric to guide the shrinking process. We provide comprehensive evidences showing that, in weight-sharing supernet, the proposed metric is more stable and accurate than accuracy-based and magnitude-based metrics to predict the capability of child models. We also show that the angle-based metric can converge fast while training supernet, enabling us to get promising shrunk search spaces efficiently. ABS can easily apply to most of popular NAS approaches (e.g. SPOS, FariNAS, ProxylessNAS, DARTS and PDARTS). Comprehensive experiments show that ABS can dramatically enhance existing NAS approaches by providing a promising shrunk search space.

READ FULL TEXT

page 5

page 12

page 17

page 19

research
11/13/2021

Towards One Shot Search Space Poisoning in Neural Architecture Search

We evaluate the robustness of a Neural Architecture Search (NAS) algorit...
research
03/22/2020

BS-NAS: Broadening-and-Shrinking One-Shot NAS with Searchable Numbers of Channels

One-Shot methods have evolved into one of the most popular methods in Ne...
research
12/05/2021

Exploring Complicated Search Spaces with Interleaving-Free Sampling

The existing neural architecture search algorithms are mostly working on...
research
04/21/2023

SSS3D: Fast Neural Architecture Search For Efficient Three-Dimensional Semantic Segmentation

We present SSS3D, a fast multi-objective NAS framework designed to find ...
research
05/07/2019

Neural Architecture Refinement: A Practical Way for Avoiding Overfitting in NAS

Neural architecture search (NAS) is proposed to automate the architectur...
research
09/15/2021

RankNAS: Efficient Neural Architecture Search by Pairwise Ranking

This paper addresses the efficiency challenge of Neural Architecture Sea...
research
03/29/2021

Rethinking Neural Operations for Diverse Tasks

An important goal of neural architecture search (NAS) is to automate-awa...

Please sign up or login with your details

Forgot password? Click here to reset