CoSDA: Continual Source-Free Domain Adaptation

04/13/2023
by   Haozhe Feng, et al.
0

Without access to the source data, source-free domain adaptation (SFDA) transfers knowledge from a source-domain trained model to target domains. Recently, SFDA has gained popularity due to the need to protect the data privacy of the source domain, but it suffers from catastrophic forgetting on the source domain due to the lack of data. To systematically investigate the mechanism of catastrophic forgetting, we first reimplement previous SFDA approaches within a unified framework and evaluate them on four benchmarks. We observe that there is a trade-off between adaptation gain and forgetting loss, which motivates us to design a consistency regularization to mitigate forgetting. In particular, we propose a continual source-free domain adaptation approach named CoSDA, which employs a dual-speed optimized teacher-student model pair and is equipped with consistency learning capability. Our experiments demonstrate that CoSDA outperforms state-of-the-art approaches in continuous adaptation. Notably, our CoSDA can also be integrated with other SFDA methods to alleviate forgetting.

READ FULL TEXT

page 2

page 7

research
07/25/2020

Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Human beings can quickly adapt to environmental changes by leveraging le...
research
04/19/2021

DANICE: Domain adaptation without forgetting in neural image compression

Neural image compression (NIC) is a new coding paradigm where coding cap...
research
07/06/2020

Dynamic memory to alleviate catastrophic forgetting in continuous learning settings

In medical imaging, technical progress or changes in diagnostic procedur...
research
01/16/2020

Continual Learning for Domain Adaptation in Chest X-ray Classification

Over the last years, Deep Learning has been successfully applied to a br...
research
07/13/2022

Domain adaptation strategies for cancer-independent detection of lymph node metastases

Recently, large, high-quality public datasets have led to the developmen...
research
05/28/2021

FReTAL: Generalizing Deepfake Detection using Knowledge Distillation and Representation Learning

As GAN-based video and image manipulation technologies become more sophi...
research
03/03/2023

EcoTTA: Memory-Efficient Continual Test-time Adaptation via Self-distilled Regularization

This paper presents a simple yet effective approach that improves contin...

Please sign up or login with your details

Forgot password? Click here to reset