From Big to Small: Adaptive Learning to Partial-Set Domains

03/14/2022
by   Zhangjie Cao, et al.
14

Domain adaptation targets at knowledge acquisition and dissemination from a labeled source domain to an unlabeled target domain under distribution shift. Still, the common requirement of identical class space shared across domains hinders applications of domain adaptation to partial-set domains. Recent advances show that deep pre-trained models of large scale endow rich knowledge to tackle diverse downstream tasks of small scale. Thus, there is a strong incentive to adapt models from large-scale domains to small-scale domains. This paper introduces Partial Domain Adaptation (PDA), a learning paradigm that relaxes the identical class space assumption to that the source class space subsumes the target class space. First, we present a theoretical analysis of partial domain adaptation, which uncovers the importance of estimating the transferable probability of each class and each instance across domains. Then, we propose Selective Adversarial Network (SAN and SAN++) with a bi-level selection strategy and an adversarial adaptation mechanism. The bi-level selection strategy up-weighs each class and each instance simultaneously for source supervised training, target self-training, and source-target adversarial adaptation through the transferable probability estimated alternately by the model. Experiments on standard partial-set datasets and more challenging tasks with superclasses show that SAN++ outperforms several domain adaptation methods.

READ FULL TEXT

page 2

page 10

page 15

research
08/10/2018

Partial Adversarial Domain Adaptation

Domain adversarial learning aligns the feature distributions across the ...
research
04/10/2020

Deep Residual Correction Network for Partial Domain Adaptation

Deep domain adaptation methods have achieved appealing performance by le...
research
07/17/2022

Coupling Adversarial Learning with Selective Voting Strategy for Distribution Alignment in Partial Domain Adaptation

In contrast to a standard closed-set domain adaptation task, partial dom...
research
05/16/2018

Extending a Parser to Distant Domains Using a Few Dozen Partially Annotated Examples

We revisit domain adaptation for parsers in the neural era. First we sho...
research
06/15/2020

Adversarial Weighting for Domain Adaptation in Regression

We present a novel instance based approach to handle regression tasks in...
research
08/16/2018

Conceptual Domain Adaptation Using Deep Learning

Deep learning has recently been shown to be instrumental in the problem ...
research
09/21/2021

The Trade-offs of Domain Adaptation for Neural Language Models

In this paper, we connect language model adaptation with concepts of mac...

Please sign up or login with your details

Forgot password? Click here to reset