Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift

by   Remi Tachet des Combes, et al.

Adversarial learning has demonstrated good performance in the unsupervised domain adaptation setting, by learning domain-invariant representations that perform well on the source domain. However, recent work has underlined limitations of existing methods in the presence of mismatched label distributions between the source and target domains. In this paper, we extend a recent upper-bound on the performance of adversarial domain adaptation to multi-class classification and more general discriminators. We then propose generalized label shift (GLS) as a way to improve robustness against mismatched label distributions. GLS states that, conditioned on the label, there exists a representation of the input that is invariant between the source and target domains. Under GLS, we provide theoretical guarantees on the transfer performance of any classifier. We also devise necessary and sufficient conditions for GLS to hold. The conditions are based on the estimation of the relative class weights between domains and on an appropriate reweighting of samples. Guided by our theoretical insights, we modify three widely used algorithms, JAN, DANN and CDAN and evaluate their performance on standard domain adaptation tasks where our method outperforms the base versions. We also demonstrate significant gains on artificially created tasks with large divergences between their source and target label distributions.


page 1

page 2

page 3

page 4


On Learning Invariant Representation for Domain Adaptation

Due to the ability of deep neural nets to learn rich representations, re...

A Label Proportions Estimation technique for Adversarial Domain Adaptation in Text Classification

Many text classification tasks are domain-dependent, and various domain ...

A Theory of Label Propagation for Subpopulation Shift

One of the central problems in machine learning is domain adaptation. Un...

Domain Adaptation under Open Set Label Shift

We introduce the problem of domain adaptation under Open Set Label Shift...

Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice

In this paper, we study the formalism of unsupervised multi-class domain...

Combating Label Distribution Shift for Active Domain Adaptation

We consider the problem of active domain adaptation (ADA) to unlabeled t...

2nd Place Solution for VisDA 2021 Challenge – Universally Domain Adaptive Image Recognition

The Visual Domain Adaptation (VisDA) 2021 Challenge calls for unsupervis...