Fast Task-Aware Architecture Inference

02/15/2019
by   Efi Kokiopoulou, et al.
0

Neural architecture search has been shown to hold great promise towards the automation of deep learning. However in spite of its potential, neural architecture search remains quite costly. To this point, we propose a novel gradient-based framework for efficient architecture search by sharing information across several tasks. We start by training many model architectures on several related (training) tasks. When a new unseen task is presented, the framework performs architecture inference in order to quickly identify a good candidate architecture, before any model is trained on the new task. At the core of our framework lies a deep value network that can predict the performance of input architectures on a task by utilizing task meta-features and the previous model training experiments performed on related tasks. We adopt a continuous parametrization of the model architecture which allows for efficient gradient-based optimization. Given a new task, an effective architecture is quickly identified by maximizing the estimated performance with respect to the model architecture parameters with simple gradient ascent. It is key to point out that our goal is to achieve reasonable performance at the lowest cost. We provide experimental results showing the effectiveness of the framework despite its high computational efficiency.

READ FULL TEXT

page 12

page 13

research
11/26/2019

Ranking architectures using meta-learning

Neural architecture search has recently attracted lots of research effor...
research
12/15/2021

Network Graph Based Neural Architecture Search

Neural architecture search enables automation of architecture design. De...
research
08/01/2018

Efficient Progressive Neural Architecture Search

This paper addresses the difficult problem of finding an optimal neural ...
research
10/14/2022

Λ-DARTS: Mitigating Performance Collapse by Harmonizing Operation Selection among Cells

Differentiable neural architecture search (DARTS) is a popular method fo...
research
06/12/2019

Continual and Multi-Task Architecture Search

Architecture search is the process of automatically learning the neural ...
research
06/11/2018

Auto-Meta: Automated Gradient Based Meta Learner Search

Fully automating machine learning pipeline is one of the outstanding cha...
research
01/27/2021

Investigating Bi-Level Optimization for Learning and Vision from a Unified Perspective: A Survey and Beyond

Bi-Level Optimization (BLO) is originated from the area of economic game...

Please sign up or login with your details

Forgot password? Click here to reset