DeepAI AI Chat
Log In Sign Up

What and Where: Learn to Plug Adapters via NAS for Multi-Domain Learning

by   Hanbin Zhao, et al.

As an important and challenging problem, multi-domain learning (MDL) typically seeks for a set of effective lightweight domain-specific adapter modules plugged into a common domain-agnostic network. Usually, existing ways of adapter plugging and structure design are handcrafted and fixed for all domains before model learning, resulting in the learning inflexibility and computational intensiveness. With this motivation, we propose to learn a data-driven adapter plugging strategy with Neural Architecture Search (NAS), which automatically determines where to plug for those adapter modules. Furthermore, we propose a NAS-adapter module for adapter structure design in a NAS-driven learning scheme, which automatically discovers effective adapter module structures for different domains. Experimental results demonstrate the effectiveness of our MDL model against existing approaches under the conditions of comparable performance. We will release the code, baselines, and training statistics for all models to facilitate future research.


page 1

page 6

page 7

page 8


Bag of Baselines for Multi-objective Joint Neural Architecture Search and Hyperparameter Optimization

Neural architecture search (NAS) and hyperparameter optimization (HPO) m...

NAS-Bench-360: Benchmarking Diverse Tasks for Neural Architecture Search

Most existing neural architecture search (NAS) benchmarks and algorithms...

Evolutionary Algorithm Enhanced Neural Architecture Search for Text-Independent Speaker Verification

State-of-the-art speaker verification models are based on deep learning ...

A Data-driven Approach to Neural Architecture Search Initialization

Algorithmic design in neural architecture search (NAS) has received a lo...

MANAS: Multi-Agent Neural Architecture Search

The Neural Architecture Search (NAS) problem is typically formulated as ...

Adversarial Branch Architecture Search for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation (UDA) is a key field in visual recognitio...