DeepAI AI Chat
Log In Sign Up

What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning

07/24/2020
by   Hanbin Zhao, et al.
29

As an important and challenging problem, multi-domain learning (MDL) typically seeks for a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in the learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with Neural Architecture Search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose a NAS-adapter module for adapter structure design in a NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance. We will release the code, baselines, and training statistics for all models to facilitate future research.

READ FULL TEXT

page 1

page 6

page 7

page 8

05/03/2021

Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization

Neural architecture search (NAS) and hyperparameter optimization (HPO) m...
10/12/2021

NAS-Bench-360: Benchmarking Diverse Tasks for Neural Architecture Search

Most existing neural architecture search (NAS) benchmarks and algorithms...
08/13/2020

Evolutionary Algorithm Enhanced Neural Architecture Search for Text-Independent Speaker Verification

State-of-the-art speaker verification models are based on deep learning ...
11/05/2021

A Data-driven Approach to Neural Architecture Search Initialization

Algorithmic design in neural architecture search (NAS) has received a lo...
09/03/2019

MANAS: Multi-Agent Neural Architecture Search

The Neural Architecture Search (NAS) problem is typically formulated as ...
02/12/2021

Adversarial Branch Architecture Search for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation (UDA) is a key field in visual recognitio...