DeepAI AI Chat
Log In Sign Up

Self-supervised Regularization for Text Classification

by   Meng Zhou, et al.

Text classification is a widely studied problem and has broad applications. In many real-world problems, the number of texts for training classification models is limited, which renders these models prone to overfitting. To address this problem, we propose SSL-Reg, a data-dependent regularization approach based on self-supervised learning (SSL). SSL is an unsupervised learning approach which defines auxiliary tasks on input data without using any human-provided labels and learns data representations by solving these auxiliary tasks. In SSL-Reg, a supervised classification task and an unsupervised SSL task are performed simultaneously. The SSL task is unsupervised, which is defined purely on input texts without using any human-provided labels. Training a model using an SSL task can prevent the model from being overfitted to a limited number of class labels in the classification task. Experiments on 17 text classification datasets demonstrate the effectiveness of our proposed method.


page 1

page 2

page 3

page 4


Contrastive Self-supervised Learning for Graph Classification

Graph classification is a widely studied problem and has broad applicati...

Disentangling Task Relations for Few-shot Text Classification via Self-Supervised Hierarchical Task Clustering

Few-Shot Text Classification (FSTC) imitates humans to learn a new text ...

A Multi-Grained Self-Interpretable Symbolic-Neural Model For Single/Multi-Labeled Text Classification

Deep neural networks based on layer-stacking architectures have historic...

DocSCAN: Unsupervised Text Classification via Learning from Neighbors

We introduce DocSCAN, a completely unsupervised text classification appr...

A cost-reducing partial labeling estimator in text classification problem

We propose a new approach to address the text classification problems wh...

Resolving label uncertainty with implicit posterior models

We propose a method for jointly inferring labels across a collection of ...

Domain Generalization by Solving Jigsaw Puzzles

Human adaptability relies crucially on the ability to learn and merge kn...