Constructing Contrastive samples via Summarization for Text Classification with limited annotations

04/11/2021
by   Yangkai Du, et al.
10

Contrastive Learning has emerged as a powerful representation learning method and facilitates various downstream tasks especially when supervised data is limited. How to construct efficient contrastive samples through data augmentation is key to its success. Unlike vision tasks, the data augmentation method for contrastive learning has not been investigated sufficiently in language tasks. In this paper, we propose a novel approach to constructing contrastive samples for language tasks using text summarization. We use these samples for supervised contrastive learning to gain better text representations which greatly benefit text classification tasks with limited annotations. To further improve the method, we mix up samples from different classes and add an extra regularization, named mix-sum regularization, in addition to the cross-entropy-loss. Experiments on real-world text classification datasets (Amazon-5, Yelp-5, AG News) demonstrate the effectiveness of the proposed contrastive learning framework with summarization-based data augmentation and mix-sum regularization.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/21/2022

Dual Contrastive Learning: Text Classification via Label-Aware Data Augmentation

Contrastive learning has achieved remarkable success in representation l...
research
05/23/2022

Conditional Supervised Contrastive Learning for Fair Text Classification

Contrastive representation learning has gained much attention due to its...
research
06/06/2022

Contrastive Graph Multimodal Model for Text Classification in Videos

The extraction of text information in videos serves as a critical step t...
research
04/22/2022

Universum-inspired Supervised Contrastive Learning

Mixup is an efficient data augmentation method which generates additiona...
research
11/23/2022

Mitigating Data Sparsity for Short Text Topic Modeling by Topic-Semantic Contrastive Learning

To overcome the data sparsity issue in short text topic modeling, existi...
research
07/02/2020

Data Augmenting Contrastive Learning of Speech Representations in the Time Domain

Contrastive Predictive Coding (CPC), based on predicting future segments...
research
09/29/2022

Few-shot Text Classification with Dual Contrastive Consistency

In this paper, we explore how to utilize pre-trained language model to p...

Please sign up or login with your details

Forgot password? Click here to reset