Source-Free Open-Set Domain Adaptation for Histopathological Images via Distilling Self-Supervised Vision Transformer

by   Guillaume Vray, et al.

There is a strong incentive to develop computational pathology models to i) ease the burden of tissue typology annotation from whole slide histological images; ii) transfer knowledge, e.g., tissue class separability from the withheld source domain to the distributionally shifted unlabeled target domain, and simultaneously iii) detect Open Set samples, i.e., unseen novel categories not present in the training source domain. This paper proposes a highly practical setting by addressing the abovementioned challenges in one fell swoop, i.e., source-free Open Set domain adaptation (SF-OSDA), which addresses the situation where a model pre-trained on the inaccessible source dataset can be adapted on the unlabeled target dataset containing Open Set samples. The central tenet of our proposed method is distilling knowledge from a self-supervised vision transformer trained in the target domain. We propose a novel style-based data augmentation used as hard positives for self-training a vision transformer in the target domain, yielding strongly contextualized embedding. Subsequently, semantically similar target images are clustered while the source model provides their corresponding weak pseudo-labels with unreliable confidence. Furthermore, we propose cluster relative maximum logit score (CRMLS) to rectify the confidence of the weak pseudo-labels and compute weighted class prototypes in the contextualized embedding space that are utilized for adapting the source model on the target domain. Our method significantly outperforms the previous methods, including open set detection, test-time adaptation, and SF-OSDA methods, setting the new state-of-the-art on three public histopathological datasets of colorectal cancer (CRC) assessment- Kather-16, Kather-19, and CRCTP. Our code is available at


page 1

page 6

page 8


Generalized Source-free Domain Adaptation

Domain adaptation (DA) aims to transfer the knowledge learned from a sou...

Test Time Transform Prediction for Open Set Histopathological Image Recognition

Tissue typology annotation in Whole Slide histological images is a compl...

On the Effectiveness of Image Rotation for Open Set Domain Adaptation

Open Set Domain Adaptation (OSDA) bridges the domain gap between a label...

Domain Adaptation without Source Data

Domain adaptation assumes that samples from source and target domains ar...

On the Robustness of Open-World Test-Time Training: Self-Training with Dynamic Prototype Expansion

Generalizing deep learning models to unknown target domain distribution ...

DATa: Domain Adaptation-Aided Deep Table Detection Using Visual-Lexical Representations

Considerable research attention has been paid to table detection by deve...

Please sign up or login with your details

Forgot password? Click here to reset