Generation-driven Contrastive Self-training for Zero-shot Text Classification with Instruction-tuned GPT

04/24/2023
by   Ruohong Zhang, et al.
0

Moreover, GPT-based zero-shot classification models tend to make independent predictions over test instances, which can be sub-optimal as the instance correlations and the decision boundaries in the target space are ignored. To address these difficulties and limitations, we propose a new approach to zero-shot text classification, namely , which leverages the strong generative power of GPT to assist in training a smaller, more adaptable, and efficient sentence encoder classifier with contrastive self-training. Specifically, GenCo applies GPT in two ways: firstly, it generates multiple augmented texts for each input instance to enhance the semantic embedding of the instance and improve the mapping to relevant labels; secondly, it generates augmented texts conditioned on the predicted label during self-training, which makes the generative process tailored to the decision boundaries in the target space. In our experiments, GenCo outperforms previous state-of-the-art methods on multiple benchmark datasets, even when only limited in-domain text data is available.

READ FULL TEXT

page 3

page 5

page 9

page 10

research
05/24/2023

PESCO: Prompt-enhanced Self Contrastive Learning for Zero-shot Text Classification

We present PESCO, a novel contrastive learning framework that substantia...
research
12/16/2021

Extreme Zero-Shot Learning for Extreme Text Classification

The eXtreme Multi-label text Classification (XMC) problem concerns findi...
research
02/11/2022

Metadata-Induced Contrastive Learning for Zero-Shot Multi-Label Text Classification

Large-scale multi-label text classification (LMTC) aims to associate a d...
research
11/29/2022

Evaluating Unsupervised Text Classification: Zero-shot and Similarity-based Approaches

Text classification of unseen classes is a challenging Natural Language ...
research
07/28/2023

WC-SBERT: Zero-Shot Text Classification via SBERT with Self-Training for Wikipedia Categories

Our research focuses on solving the zero-shot text classification proble...
research
10/23/2022

Conformal Predictor for Improving Zero-shot Text Classification Efficiency

Pre-trained language models (PLMs) have been shown effective for zero-sh...
research
07/13/2023

AvatarFusion: Zero-shot Generation of Clothing-Decoupled 3D Avatars Using 2D Diffusion

Large-scale pre-trained vision-language models allow for the zero-shot t...

Please sign up or login with your details

Forgot password? Click here to reset