Not all domains are equally complex: Adaptive Multi-Domain Learning

03/25/2020
by   Ali Senhaji, et al.
0

Deep learning approaches are highly specialized and require training separate models for different tasks. Multi-domain learning looks at ways to learn a multitude of different tasks, each coming from a different domain, at once. The most common approach in multi-domain learning is to form a domain agnostic model, the parameters of which are shared among all domains, and learn a small number of extra domain-specific parameters for each individual new domain. However, different domains come with different levels of difficulty; parameterizing the models of all domains using an augmented version of the domain agnostic model leads to unnecessarily inefficient solutions, especially for easy to solve tasks. We propose an adaptive parameterization approach to deep neural networks for multi-domain learning. The proposed approach performs on par with the original approach while reducing by far the number of parameters, leading to efficient multi-domain learning solutions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/03/2019

Depthwise Convolution is All You Need for Learning Multiple Visual Domains

There is a growing interest in designing models that can deal with image...
research
05/11/2017

Incremental Learning Through Deep Adaptation

Given an existing trained neural network, it is often desirable to be ab...
research
09/19/2023

Decoupled Training: Return of Frustratingly Easy Multi-Domain Learning

Multi-domain learning (MDL) aims to train a model with minimal average r...
research
03/27/2018

Efficient parametrization of multi-domain deep neural networks

A practical limitation of deep neural networks is their high degree of s...
research
01/27/2021

One Model to Serve All: Star Topology Adaptive Recommender for Multi-Domain CTR Prediction

Traditional industrial recommenders are usually trained on a single busi...
research
05/15/2019

Budget-Aware Adapters for Multi-Domain Learning

Multi-Domain Learning (MDL) refers to the problem of learning a set of m...
research
10/14/2022

Parameter Sharing in Budget-Aware Adapters for Multi-Domain Learning

Deep learning has achieved state-of-the-art performance on several compu...

Please sign up or login with your details

Forgot password? Click here to reset