Beyond ℋ-Divergence: Domain Adaptation Theory With Jensen-Shannon Divergence

07/30/2020
by   Changjian Shui, et al.
5

We reveal the incoherence between the widely-adopted empirical domain adversarial training and its generally-assumed theoretical counterpart based on ℋ-divergence. Concretely, we find that ℋ-divergence is not equivalent to Jensen-Shannon divergence, the optimization objective in domain adversarial training. To this end, we establish a new theoretical framework by directly proving the upper and lower target risk bounds based on joint distributional Jensen-Shannon divergence. We further derive bi-directional upper bounds for marginal and conditional shifts. Our framework exhibits inherent flexibilities for different transfer learning problems, which is usable for various scenarios where ℋ-divergence-based theory fails to adapt. From an algorithmic perspective, our theory enables a generic guideline unifying principles of semantic conditional matching, feature marginal matching, and label marginal shift correction. We employ algorithms for each principle and empirically validate the benefits of our framework on real datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/18/2020

Joint Contrastive Learning for Unsupervised Domain Adaptation

Enhancing feature transferability by matching marginal distributions has...
research
10/13/2020

Information-Theoretic Bounds on Transfer Generalization Gap Based on Jensen-Shannon Divergence

In transfer learning, training and testing data sets are drawn from diff...
research
07/28/2021

Adversarial Unsupervised Domain Adaptation with Conditional and Label Shift: Infer, Align and Iterate

In this work, we propose an adversarial unsupervised domain adaptation (...
research
04/27/2020

Maximum Density Divergence for Domain Adaptation

Unsupervised domain adaptation addresses the problem of transferring kno...
research
07/22/2021

Domain Generalization under Conditional and Label Shifts via Variational Bayesian Inference

In this work, we propose a domain generalization (DG) approach to learn ...
research
12/14/2020

REDAT: Accent-Invariant Representation for End-to-End ASR by Domain Adversarial Training with Relabeling

Accents mismatching is a critical problem for end-to-end ASR. This paper...
research
06/17/2019

MixUp as Directional Adversarial Training

In this work, we explain the working mechanism of MixUp in terms of adve...

Please sign up or login with your details

Forgot password? Click here to reset