Efficient Domain Generalization via Common-Specific Low-Rank Decomposition

03/28/2020
by   Vihari Piratla, et al.
0

Domain generalization refers to the task of training a model which generalizes to new domains that are not seen during training. We present CSD (Common Specific Decomposition), for this setting,which jointly learns a common component (which generalizes to new domains) and a domain specific component (which overfits on training domains). The domain specific components are discarded after training and only the common component is retained. The algorithm is extremely simple and involves only modifying the final linear classification layer of any given neural network architecture. We present a principled analysis to understand existing approaches, provide identifiability results of CSD,and study effect of low-rank on domain generalization. We show that CSD either matches or beats state of the art approaches for domain generalization based on domain erasure, domain perturbed data augmentation, and meta-learning. Further diagnostics on rotated MNIST, where domains are interpretable, confirm the hypothesis that CSD successfully disentangles common and domain specific components and hence leads to better domain generalization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/18/2021

Exploiting Domain-Specific Features to Enhance Domain Generalization

Domain Generalization (DG) aims to train a model, from multiple observed...
research
07/09/2018

Domain2Vec: Deep Domain Generalization

We address the problem of domain generalization where a decision functio...
research
05/20/2022

Set-based Meta-Interpolation for Few-Task Meta-Learning

Meta-learning approaches enable machine learning systems to adapt to new...
research
07/14/2023

DISPEL: Domain Generalization via Domain-Specific Liberating

Domain generalization aims to learn a generalization model that can perf...
research
04/03/2020

Sequential Learning for Domain Generalization

In this paper we propose a sequential learning framework for Domain Gene...
research
03/24/2022

FAMLP: A Frequency-Aware MLP-Like Architecture For Domain Generalization

MLP-like models built entirely upon multi-layer perceptrons have recentl...
research
05/01/1997

Connectionist Theory Refinement: Genetically Searching the Space of Network Topologies

An algorithm that learns from a set of examples should ideally be able t...

Please sign up or login with your details

Forgot password? Click here to reset