Variational Model Perturbation for Source-Free Domain Adaptation

10/19/2022
by   Mengmeng Jing, et al.
5

We aim for source-free domain adaptation, where the task is to deploy a model pre-trained on source domains to target domains. The challenges stem from the distribution shift from the source to the target domain, coupled with the unavailability of any source data and labeled target data for optimization. Rather than fine-tuning the model by updating the parameters, we propose to perturb the source model to achieve adaptation to target domains. We introduce perturbations into the model parameters by variational Bayesian inference in a probabilistic framework. By doing so, we can effectively adapt the model to the target domain while largely preserving the discriminative ability. Importantly, we demonstrate the theoretical connection to learning Bayesian neural networks, which proves the generalizability of the perturbed model to target domains. To enable more efficient optimization, we further employ a parameter sharing strategy, which substantially reduces the learnable parameters compared to a fully Bayesian neural network. Our model perturbation provides a new probabilistic way for domain adaptation which enables efficient adaptation to target domains while maximally preserving knowledge in source models. Experiments on several source-free benchmarks under three different evaluation settings verify the effectiveness of the proposed variational model perturbation for source-free domain adaptation.

READ FULL TEXT
research
12/15/2022

Co-Learning with Pre-Trained Networks Improves Source-Free Domain Adaptation

Source-free domain adaptation aims to adapt a source model trained on fu...
research
09/02/2021

On-target Adaptation

Domain adaptation seeks to mitigate the shift between training on the so...
research
07/25/2019

Unsupervised Domain Adaptation via Calibrating Uncertainties

Unsupervised domain adaptation (UDA) aims at inferring class labels for ...
research
08/16/2022

Uncertainty-guided Source-free Domain Adaptation

Source-free domain adaptation (SFDA) aims to adapt a classifier to an un...
research
01/14/2020

Domain Adaption for Knowledge Tracing

With the rapid development of online education system, knowledge tracing...
research
08/30/2022

Super-model ecosystem: A domain-adaptation perspective

This paper attempts to establish the theoretical foundation for the emer...
research
05/25/2017

Principled Hybrids of Generative and Discriminative Domain Adaptation

We propose a probabilistic framework for domain adaptation that blends b...

Please sign up or login with your details

Forgot password? Click here to reset