A Two-Stage Framework with Self-Supervised Distillation For Cross-Domain Text Classification

04/18/2023
by   Yunlong Feng, et al.
0

Cross-domain text classification aims to adapt models to a target domain that lacks labeled data. It leverages or reuses rich labeled data from the different but related source domain(s) and unlabeled data from the target domain. To this end, previous work focuses on either extracting domain-invariant features or task-agnostic features, ignoring domain-aware features that may be present in the target domain and could be useful for the downstream task. In this paper, we propose a two-stage framework for cross-domain text classification. In the first stage, we finetune the model with mask language modeling (MLM) and labeled data from the source domain. In the second stage, we further fine-tune the model with self-supervised distillation (SSD) and unlabeled data from the target domain. We evaluate its performance on a public cross-domain text classification benchmark and the experiment results show that our method achieves new state-of-the-art results for both single-source domain adaptations (94.17 ↑1.34

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/02/2018

Transductive Learning with String Kernels for Cross-Domain Text Classification

For many text classification tasks, there is a major problem posed by th...
research
09/24/2020

Feature Adaptation of Pre-Trained Language Models across Languages and Domains for Text Classification

Adapting pre-trained language models (PrLMs) (e.g., BERT) to new domains...
research
09/16/2018

Cross-Domain Labeled LDA for Cross-Domain Text Classification

Cross-domain text classification aims at building a classifier for a tar...
research
10/23/2022

Unsupervised Non-transferable Text Classification

Training a good deep learning model requires substantial data and comput...
research
06/20/2022

Domain-Adaptive Text Classification with Structured Knowledge from Unlabeled Data

Domain adaptive text classification is a challenging problem for the lar...
research
03/16/2021

Consistent Posterior Distributions under Vessel-Mixing: A Regularization for Cross-Domain Retinal Artery/Vein Classification

Retinal artery/vein (A/V) classification is a critical technique for dia...
research
11/30/2017

Graph Distillation for Action Detection with Privileged Information

In this work, we propose a technique that tackles the video understandin...

Please sign up or login with your details

Forgot password? Click here to reset