DCLP: Neural Architecture Predictor with Curriculum Contrastive Learning

02/25/2023
by   Shenghe Zheng, et al.
0

Neural predictors currently show great potential in the performance evaluation phase of neural architecture search (NAS). Despite their efficiency in the evaluation process, it is challenging to train the predictor with fewer architecture evaluations for efficient NAS. However, most of the current approaches are more concerned with improving the structure of the predictor to solve this problem, while the full use of the information contained in unlabeled data is less explored. To address this issue, we introduce a contrastive learning framework with curriculum learning guidance for the neural predictor called DCLP. To be specific, we develop a plan for the training order of positive samples during pre-training through the proposed difficulty measurer and training scheduler, and utilize the contrastive learner to learn representations of data. Compared with existing predictors, we experimentally demonstrate that DCLP has high accuracy and efficiency, and also shows an encouraging ability to discover superior architectures in multiple search spaces when combined with search strategies.

READ FULL TEXT

page 2

page 7

page 11

research
02/21/2023

A General-Purpose Transferable Predictor for Neural Architecture Search

Understanding and modelling the performance of neural architectures is k...
research
03/08/2021

Contrastive Neural Architecture Search with Neural Architecture Comparators

One of the key steps in Neural Architecture Search (NAS) is to estimate ...
research
10/31/2020

Self-supervised Representation Learning for Evolutionary Neural Architecture Search

Recently proposed neural architecture search (NAS) algorithms adopt neur...
research
07/28/2021

Homogeneous Architecture Augmentation for Neural Predictor

Neural Architecture Search (NAS) can automatically design well-performed...
research
11/30/2022

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

Predicting neural architecture performance is a challenging task and is ...
research
07/16/2023

Neural Architecture Retrieval

With the increasing number of new neural architecture designs and substa...
research
02/04/2022

Heed the Noise in Performance Evaluations in Neural Architecture Search

Neural Architecture Search (NAS) has recently become a topic of great in...

Please sign up or login with your details

Forgot password? Click here to reset