Discretization-Aware Architecture Search

07/07/2020
by   Yunjie Tian, et al.
9

The search cost of neural architecture search (NAS) has been largely reduced by weight-sharing methods. These methods optimize a super-network with all possible edges and operations, and determine the optimal sub-network by discretization, i.e., pruning off weak candidates. The discretization process, performed on either operations or edges, incurs significant inaccuracy and thus the quality of the final architecture is not guaranteed. This paper presents discretization-aware architecture search (DA2S), with the core idea being adding a loss term to push the super-network towards the configuration of desired topology, so that the accuracy loss brought by discretization is largely alleviated. Experiments on standard image classification benchmarks demonstrate the superiority of our approach, in particular, under imbalanced target network configurations that were not studied before.

READ FULL TEXT
research
07/12/2023

DDNAS: Discretized Differentiable Neural Architecture Search for Text Classification

Neural Architecture Search (NAS) has shown promising capability in learn...
research
05/19/2022

A Hardware-Aware Framework for Accelerating Neural Architecture Search Across Modalities

Recent advances in Neural Architecture Search (NAS) such as one-shot NAS...
research
09/03/2019

MANAS: Multi-Agent Neural Architecture Search

The Neural Architecture Search (NAS) problem is typically formulated as ...
research
08/25/2021

iDARTS: Improving DARTS by Node Normalization and Decorrelation Discretization

Differentiable ARchiTecture Search (DARTS) uses a continuous relaxation ...
research
02/25/2022

A Hardware-Aware System for Accelerating Deep Neural Network Optimization

Recent advances in Neural Architecture Search (NAS) which extract specia...
research
06/22/2021

On Constrained Optimization in Differentiable Neural Architecture Search

Differentiable Architecture Search (DARTS) is a recently proposed neural...
research
09/30/2022

IMB-NAS: Neural Architecture Search for Imbalanced Datasets

Class imbalance is a ubiquitous phenomenon occurring in real world data ...

Please sign up or login with your details

Forgot password? Click here to reset