Improve Ranking Correlation of Super-net through Training Scheme from One-shot NAS to Few-shot NAS

06/13/2022
by   Jiawei Liu, et al.
0

The algorithms of one-shot neural architecture search(NAS) have been widely used to reduce computation consumption. However, because of the interference among the subnets in which weights are shared, the subnets inherited from these super-net trained by those algorithms have poor consistency in precision ranking. To address this problem, we propose a step-by-step training super-net scheme from one-shot NAS to few-shot NAS. In the training scheme, we firstly train super-net in a one-shot way, and then we disentangle the weights of super-net by splitting them into multi-subnets and training them gradually. Finally, our method ranks 4th place in the CVPR2022 3rd Lightweight NAS Challenge Track1. Our code is available at https://github.com/liujiawei2333/CVPR2022-NAS-competition-Track-1-4th-solution.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/27/2022

Prior-Guided One-shot Neural Architecture Search

Neural architecture search methods seek optimal candidates with efficien...
research
11/23/2022

NAS-LID: Efficient Neural Architecture Search with Local Intrinsic Dimension

One-shot neural architecture search (NAS) substantially improves the sea...
research
06/11/2021

K-shot NAS: Learnable Weight-Sharing for NAS with K-shot Supernets

In one-shot weight sharing for NAS, the weights of each operation (at ea...
research
02/28/2023

PA DA: Jointly Sampling PAth and DAta for Consistent NAS

Based on the weight-sharing mechanism, one-shot NAS methods train a supe...
research
05/02/2023

Predict NAS Multi-Task by Stacking Ensemble Models using GP-NAS

Accurately predicting the performance of architecture with small sample ...
research
01/24/2023

RD-NAS: Enhancing One-shot Supernet Ranking Ability via Ranking Distillation from Zero-cost Proxies

Neural architecture search (NAS) has made tremendous progress in the aut...
research
09/24/2019

Understanding and Improving One-shot Neural Architecture Optimization

The ability of accurately ranking candidate architectures is the key to ...

Please sign up or login with your details

Forgot password? Click here to reset