Support and Invertibility in Domain-Invariant Representations

03/08/2019
by   Fredrik D. Johansson, et al.
10

Learning domain-invariant representations has become a popular approach to unsupervised domain adaptation and is often justified by invoking a particular suite of theoretical results. We argue that there are two significant flaws in such arguments. First, the results in question hold only for a fixed representation and do not account for information lost in non-invertible transformations. Second, domain invariance is often a far too strict requirement and does not always lead to consistent estimation, even under strong and favorable assumptions. In this work, we give generalization bounds for unsupervised domain adaptation that hold for any representation function by acknowledging the cost of non-invertibility. In addition, we show that penalizing distance between densities is often wasteful and propose a bound based on measuring the extent to which the support of the source domain covers the target domain. We perform experiments on well-known benchmarks that illustrate the short-comings of current standard practice.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/11/2018

Unsupervised Domain Adaptation Based on Source-guided Discrepancy

Unsupervised domain adaptation is the problem setting where data generat...
research
10/13/2019

The Role of Embedding Complexity in Domain-invariant Representations

Unsupervised domain adaptation aims to generalize the hypothesis trained...
research
06/24/2020

Robust Domain Adaptation: Representations, Weights and Inductive Bias

Unsupervised Domain Adaptation (UDA) has attracted a lot of attention in...
research
09/16/2021

On the inductive biases of deep domain adaptation

Domain alignment is currently the most prevalent solution to unsupervise...
research
06/25/2020

Target Consistency for Domain Adaptation: when Robustness meets Transferability

Learning Invariant Representations has been successfully applied for rec...
research
06/23/2022

Measuring the Feasibility of Analogical Transfer using Complexity

Analogies are 4-ary relations of the form "A is to B as C is to D". Whil...
research
12/21/2022

Learning List-Level Domain-Invariant Representations for Ranking

Domain adaptation aims to transfer the knowledge acquired by models trai...

Please sign up or login with your details

Forgot password? Click here to reset