Rethinking Neural Operations for Diverse Tasks

by   Nicholas Roberts, et al.

An important goal of neural architecture search (NAS) is to automate-away the design of neural networks on new tasks in under-explored domains. Motivated by this broader vision for NAS, we study the problem of enabling users to discover the right neural operations given data from their specific domain. We introduce a search space of neural operations called XD-Operations that mimic the inductive bias of standard multichannel convolutions while being much more expressive: we prove that XD-operations include many named operations across several application areas. Starting with any standard backbone network such as LeNet or ResNet, we show how to transform it into an architecture search space over XD-operations and how to traverse the space using a simple weight-sharing scheme. On a diverse set of applications–image classification, solving partial differential equations (PDEs), and sequence modeling–our approach consistently yields models with lower error than baseline networks and sometimes even lower error than expert-designed domain-specific approaches.


Efficient Architecture Search for Diverse Tasks

While neural architecture search (NAS) has enabled automated machine lea...

Angle-based Search Space Shrinking for Neural Architecture Search

In this work, we present a simple and general search space shrinking met...

NASIB: Neural Architecture Search withIn Budget

Neural Architecture Search (NAS) represents a class of methods to genera...

Efficient Backbone Search for Scene Text Recognition

Scene text recognition (STR) is very challenging due to the diversity of...

ISNAS-DIP: Image-Specific Neural Architecture Search for Deep Image Prior

Recent works show that convolutional neural network (CNN) architectures ...

Adaptive Interaction Modeling via Graph Operations Search

Interaction modeling is important for video action analysis. Recently, s...

AutoRC: Improving BERT Based Relation Classification Models via Architecture Search

Although BERT based relation classification (RC) models have achieved si...

Code Repositories