Transfer NAS: Knowledge Transfer between Search Spaces with Transformer Agents

06/19/2019
by   Zalán Borsos, et al.
0

Recent advances in Neural Architecture Search (NAS) have produced state-of-the-art architectures on several tasks. NAS shifts the efforts of human experts from developing novel architectures directly to designing architecture search spaces and methods to explore them efficiently. The search space definition captures prior knowledge about the properties of the architectures and it is crucial for the complexity and the performance of the search algorithm. However, different search space definitions require restarting the learning process from scratch. We propose a novel agent based on the Transformer that supports joint training and efficient transfer of prior knowledge between multiple search spaces and tasks.

READ FULL TEXT
research
05/08/2022

αNAS: Neural Architecture Search using Property Guided Synthesis

In the past few years, neural architecture search (NAS) has become an in...
research
11/03/2022

Towards Discovering Neural Architectures from Scratch

The discovery of neural architectures from scratch is the long-standing ...
research
09/16/2019

Pose Neural Fabrics Search

Neural Architecture Search (NAS) technologies have been successfully per...
research
05/19/2021

Efficient Transfer Learning via Joint Adaptation of Network Architecture and Weight

Transfer learning can boost the performance on the targettask by leverag...
research
03/10/2022

Towards Less Constrained Macro-Neural Architecture Search

Networks found with Neural Architecture Search (NAS) achieve state-of-th...
research
12/01/2021

Training BatchNorm Only in Neural Architecture Search and Beyond

This work investigates the usage of batch normalization in neural archit...
research
07/31/2020

HMCNAS: Neural Architecture Search using Hidden Markov Chains and Bayesian Optimization

Neural Architecture Search has achieved state-of-the-art performance in ...

Please sign up or login with your details

Forgot password? Click here to reset