A General Upper Bound for Unsupervised Domain Adaptation

10/03/2019
by   Dexuan Zhang, et al.
15

In this work, we present a novel upper bound of target error to address the problem for unsupervised domain adaptation. Recent studies reveal that a deep neural network can learn transferable features which generalize well to novel tasks. Furthermore, a theory proposed by Ben-David et al. (2010) provides a upper bound for target error when transferring the knowledge, which can be summarized as minimizing the source error and distance between marginal distributions simultaneously. However, common methods based on the theory usually ignore the joint error such that samples from different classes might be mixed together when matching marginal distribution. And in such case, no matter how we minimize the marginal discrepancy, the target error is not bounded due to an increasing joint error. To address this problem, we propose a general upper bound taking joint error into account, such that the undesirable case can be properly penalized. In addition, we utilize constrained hypothesis space to further formalize a tighter bound as well as a novel cross margin discrepancy to measure the dissimilarity between hypotheses which alleviates instability during adversarial learning. Extensive empirical evidence shows that our proposal outperforms related approaches in image classification error rates on standard domain adaptation benchmarks.

READ FULL TEXT

page 18

page 20

research
03/07/2022

Maximizing Conditional Independence for Unsupervised Domain Adaptation

Unsupervised domain adaptation studies how to transfer a learner from a ...
research
06/18/2020

Joint Contrastive Learning for Unsupervised Domain Adaptation

Enhancing feature transferability by matching marginal distributions has...
research
07/17/2017

PAC-Bayes and Domain Adaptation

We provide two main contributions in PAC-Bayesian theory for domain adap...
research
05/26/2019

Learning Smooth Representation for Unsupervised Domain Adaptation

In unsupervised domain adaptation, existing methods utilizing the bounda...
research
03/14/2022

Bures Joint Distribution Alignment with Dynamic Margin for Unsupervised Domain Adaptation

Unsupervised domain adaptation (UDA) is one of the prominent tasks of tr...
research
07/05/2022

Cooperative Distribution Alignment via JSD Upper Bound

Unsupervised distribution alignment estimates a transformation that maps...
research
12/30/2020

How does the Combined Risk Affect the Performance of Unsupervised Domain Adaptation Approaches?

Unsupervised domain adaptation (UDA) aims to train a target classifier w...

Please sign up or login with your details

Forgot password? Click here to reset