Smooth Variational Graph Embeddings for Efficient Neural Architecture Search

10/09/2020
by   Jovita Lukasik, et al.
0

In this paper, we propose an approach to neural architecture search (NAS) based on graph embeddings. NAS has been addressed previously using discrete, sampling based methods, which are computationally expensive as well as differentiable approaches, which come at lower costs but enforce stronger constraints on the search space. The proposed approach leverages advantages from both sides by building a smooth variational neural architecture embedding space in which we evaluate a structural subset of architectures at training time using the predicted performance while it allows to extrapolate from this subspace at inference time. We evaluate the proposed approach in the context of two common search spaces, the graph structure defined by the ENAS approach and the NAS-Bench-101 search space, and improve over the state of the art in both.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/27/2018

Neural Architecture Search Over a Graph Search Space

Neural architecture search (NAS) enabled the discovery of state-of-the-a...
research
02/08/2021

Contrastive Embeddings for Neural Architectures

The performance of algorithms for neural architecture search strongly de...
research
09/03/2019

MANAS: Multi-Agent Neural Architecture Search

The Neural Architecture Search (NAS) problem is typically formulated as ...
research
12/11/2019

A Variational-Sequential Graph Autoencoder for Neural Architecture Performance Prediction

In computer vision research, the process of automating architecture engi...
research
08/17/2022

Field-wise Embedding Size Search via Structural Hard Auxiliary Mask Pruning for Click-Through Rate Prediction

Feature embeddings are one of the most essential steps when training dee...
research
05/13/2021

Compatibility-aware Heterogeneous Visual Search

We tackle the problem of visual search under resource constraints. Exist...
research
10/18/2017

Minimizing Task Space Frechet Error via Efficient Incremental Graph Search

We present an algorithm that generates a collision-free configuration-sp...

Please sign up or login with your details

Forgot password? Click here to reset