Label Confusion Learning to Enhance Text Classification Models

by   Biyang Guo, et al.

Representing a true label as a one-hot vector is a common practice in training text classification models. However, the one-hot representation may not adequately reflect the relation between the instances and labels, as labels are often not completely independent and instances may relate to multiple labels in practice. The inadequate one-hot representations tend to train the model to be over-confident, which may result in arbitrary prediction and model overfitting, especially for confused datasets (datasets with very similar labels) or noisy datasets (datasets with labeling errors). While training models with label smoothing (LS) can ease this problem in some degree, it still fails to capture the realistic relation among labels. In this paper, we propose a novel Label Confusion Model (LCM) as an enhancement component to current popular text classification models. LCM can learn label confusion to capture semantic overlap among labels by calculating the similarity between instances and labels during training and generate a better label distribution to replace the original one-hot label vector, thus improving the final classification performance. Extensive experiments on five text classification benchmark datasets reveal the effectiveness of LCM for several widely used deep learning classification models. Further experiments also verify that LCM is especially helpful for confused or noisy datasets and superior to the label smoothing method.


Structure-Aware Label Smoothing for Graph Neural Networks

Representing a label distribution as a one-hot vector is a common practi...

Improving Model Training via Self-learned Label Representations

Modern neural network architectures have shown remarkable success in sev...

Leaf Cultivar Identification via Prototype-enhanced Learning

Plant leaf identification is crucial for biodiversity protection and con...

Training and Prediction Data Discrepancies: Challenges of Text Classification with Noisy, Historical Data

Industry datasets used for text classification are rarely created for th...

The Missing Link: Finding label relations across datasets

Computer Vision is driven by the many datasets which can be used for tra...

Locally Adaptive Label Smoothing for Predictive Churn

Training modern neural networks is an inherently noisy process that can ...

Benchmarking Popular Classification Models' Robustness to Random and Targeted Corruptions

Text classification models, especially neural networks based models, hav...

Please sign up or login with your details

Forgot password? Click here to reset