Prompt-Learning for Short Text Classification

02/23/2022
by   Yi Zhu, et al.
1

In the short text, the extreme short length, feature sparsity and high ambiguity pose huge challenge to classification tasks. Recently, as an effective method for tuning Pre-trained Language Models for specific downstream tasks, prompt-learning has attracted vast amount of attention and research. The main intuition behind the prompt-learning is to insert template into the input and convert the text classification tasks into equivalent cloze-style tasks. However, most prompt-learning methods expand label words manually or only consider the class name for knowledge incorporating in cloze-style prediction, which will inevitably incurred omissions and bias in classification tasks. In this paper, we propose a simple short text classification approach that makes use of prompt-learning based on knowledgeable expansion, which can consider both the short text itself and class name during expanding label words space. Specifically, the top N concepts related to the entity in short text are retrieved from the open Knowledge Graph like Probase, and we further refine the expanded label words by the distance calculation between selected concepts and class label. Experimental results show that our approach obtains obvious improvement compared with other fine-tuning, prompt-learning and knowledgeable prompt-tuning methods, outperforming the state-of-the-art by up to 6 Accuracy points on three well-known datasets.

READ FULL TEXT
research
08/04/2021

Knowledgeable Prompt-tuning: Incorporating Knowledge into Prompt Verbalizer for Text Classification

Tuning pre-trained language models (PLMs) with task-specific prompts has...
research
06/15/2019

Towards Integration of Statistical Hypothesis Tests into Deep Neural Networks

We report our ongoing work about a new deep architecture working in tand...
research
09/26/2019

Pre-train, Interact, Fine-tune: A Novel Interaction Representation for Text Classification

Text representation can aid machines in understanding text. Previous wor...
research
08/07/2019

A Simple and Effective Approach for Fine Tuning Pre-trained Word Embeddings for Improved Text Classification

This work presents a new and simple approach for fine-tuning pretrained ...
research
05/10/2022

Sibylvariant Transformations for Robust Text Classification

The vast majority of text transformation techniques in NLP are inherentl...
research
11/30/2022

Learning Label Modular Prompts for Text Classification in the Wild

Machine learning models usually assume i.i.d data during training and te...
research
09/16/2019

Short-Text Classification Using Unsupervised Keyword Expansion

Short-text classification, like all data science, struggles to achieve h...

Please sign up or login with your details

Forgot password? Click here to reset