Rethinking Neural Operations for Diverse Tasks

03/29/2021
by   Nicholas Roberts, et al.
7

An important goal of neural architecture search (NAS) is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this broader vision for NAS, we study the problem of enabling users to discover the right neural operations given data from their specific domain. We introduce a search space of neural operations called XD-Operations that mimic the inductive bias of standard multichannel convolutions while being much more expressive: we prove that XD-operations include many named operations across several application areas. Starting with any standard backbone network such as LeNet or ResNet, we show how to transform it into an architecture search space over XD-operations and how to traverse the space using a simple weight-sharing scheme. On a diverse set of applications–image classification, solving partial differential equations (PDEs), and sequence modeling–our approach consistently yields models with lower error than baseline networks and sometimes even lower error than expert-designed domain-specific approaches.

READ FULL TEXT
04/15/2022

Efficient Architecture Search for Diverse Tasks

While neural architecture search (NAS) has enabled automated machine lea...
04/28/2020

Angle-based Search Space Shrinking for Neural Architecture Search

In this work, we present a simple and general search space shrinking met...
10/19/2019

NASIB: Neural Architecture Search withIn Budget

Neural Architecture Search (NAS) represents a class of methods to genera...
03/14/2020

Efficient Backbone Search for Scene Text Recognition

Scene text recognition (STR) is very challenging due to the diversity of...
11/27/2021

ISNAS-DIP: Image-Specific Neural Architecture Search for Deep Image Prior

Recent works show that convolutional neural network (CNN) architectures ...
05/05/2020

Adaptive Interaction Modeling via Graph Operations Search

Interaction modeling is important for video action analysis. Recently, s...
09/22/2020

AutoRC: Improving BERT Based Relation Classification Models via Architecture Search

Although BERT based relation classification (RC) models have achieved si...

Code Repositories