Connect, Not Collapse: Explaining Contrastive Learning for Unsupervised Domain Adaptation

by   Kendrick Shen, et al.
Stanford University

We consider unsupervised domain adaptation (UDA), where labeled data from a source domain (e.g., photographs) and unlabeled data from a target domain (e.g., sketches) are used to learn a classifier for the target domain. Conventional UDA methods (e.g., domain adversarial training) learn domain-invariant features to improve generalization to the target domain. In this paper, we show that contrastive pre-training, which learns features on unlabeled source and target data and then fine-tunes on labeled source data, is competitive with strong UDA methods. However, we find that contrastive pre-training does not learn domain-invariant features, diverging from conventional UDA intuitions. We show theoretically that contrastive pre-training can learn features that vary subtantially across domains but still generalize to the target domain, by disentangling domain and class information. Our results suggest that domain invariance is not necessary for UDA. We empirically validate our theory on benchmark vision datasets.


page 1

page 2

page 3

page 4


Open-Set Domain Adaptation with Visual-Language Foundation Models

Unsupervised domain adaptation (UDA) has proven to be very effective in ...

The Role of Embedding Complexity in Domain-invariant Representations

Unsupervised domain adaptation aims to generalize the hypothesis trained...

On the inductive biases of deep domain adaptation

Domain alignment is currently the most prevalent solution to unsupervise...

Unsupervised Domain Adaptation with Contrastive Learning for OCT Segmentation

Accurate segmentation of retinal fluids in 3D Optical Coherence Tomograp...

Domain Adaptation via Prompt Learning

Unsupervised domain adaption (UDA) aims to adapt models learned from a w...

Spectral Unsupervised Domain Adaptation for Visual Recognition

Unsupervised domain adaptation (UDA) aims to learn a well-performed mode...

Please sign up or login with your details

Forgot password? Click here to reset