Pretraining Neural Architecture Search Controllers with Locality-based Self-Supervised Learning

03/15/2021
by   Kwanghee Choi, et al.
0

Neural architecture search (NAS) has fostered various fields of machine learning. Despite its prominent dedications, many have criticized the intrinsic limitations of high computational cost. We aim to ameliorate this by proposing a pretraining scheme that can be generally applied to controller-based NAS. Our method, locality-based self-supervised classification task, leverages the structural similarity of network architectures to obtain good architecture representations. We incorporate our method into neural architecture optimization (NAO) to analyze the pretrained embeddings and its effectiveness and highlight that adding metric learning loss brings a favorable impact on NAS. Our code is available at <https://github.com/Multi-Objective-NAS/self-supervised-nas>.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

05/15/2022

Proxyless Neural Architecture Adaptation for Supervised Learning and Self-Supervised Learning

Recently, Neural Architecture Search (NAS) methods have been introduced ...
07/03/2020

Self-supervised Neural Architecture Search

Neural Architecture Search (NAS) has been used recently to achieve impro...
02/24/2020

Semi-Supervised Neural Architecture Search

Neural architecture search (NAS) relies on a good controller to generate...
09/17/2021

Self-Supervised Neural Architecture Search for Imbalanced Datasets

Neural Architecture Search (NAS) provides state-of-the-art results when ...
10/08/2018

NSGA-NET: A Multi-Objective Genetic Algorithm for Neural Architecture Search

This paper introduces NSGA-Net, an evolutionary approach for neural arch...
02/21/2021

Contrastive Self-supervised Neural Architecture Search

This paper proposes a novel cell-based neural architecture search algori...
01/19/2021

Learning Efficient, Explainable and Discriminative Representations for Pulmonary Nodules Classification

Automatic pulmonary nodules classification is significant for early diag...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.