DeepAI AI Chat
Log In Sign Up

Text Classification Using Label Names Only: A Language Model Self-Training Approach

by   Yu Meng, et al.
University of Illinois at Urbana-Champaign
Georgia Institute of Technology

Current text classification methods typically require a good number of human-labeled documents as training data, which can be costly and difficult to obtain in real applications. Humans can perform classification without seeing any labeled examples but only based on a small set of words describing the categories to be classified. In this paper, we explore the potential of only using the label name of each class to train classification models on unlabeled data, without using any labeled documents. We use pre-trained neural language models both as general linguistic knowledge sources for category understanding and as representation learning models for document classification. Our method (1) associates semantically related words with the label names, (2) finds category-indicative words and trains the model to predict their implied categories, and (3) generalizes the model via self-training. We show that our model achieves around 90 and sentiment classification without using any labeled documents but learning from unlabeled data supervised by at most 3 words (1 in most cases) per class as the label name.


page 1

page 2

page 3

page 4


Improving Probabilistic Models in Text Classification via Active Learning

When using text data, social scientists often classify documents in orde...

MotifClass: Weakly Supervised Text Classification with Higher-order Metadata Information

We study the problem of weakly supervised text classification, which aim...

A probabilistic methodology for multilabel classification

Multilabel classification is a relatively recent subfield of machine lea...

Out-of-Category Document Identification Using Target-Category Names as Weak Supervision

Identifying outlier documents, whose content is different from the major...

Robustifying Sentiment Classification by Maximally Exploiting Few Counterfactuals

For text classification tasks, finetuned language models perform remarka...

Exploring Distantly-Labeled Rationales in Neural Network Models

Recent studies strive to incorporate various human rationales into neura...

Few-Shot Text Classification with Pre-Trained Word Embeddings and a Human in the Loop

Most of the literature around text classification treats it as a supervi...