LightXML: Transformer with Dynamic Negative Sampling for High-Performance Extreme Multi-label Text Classification

01/09/2021
by   Ting Jiang, et al.
0

Extreme Multi-label text Classification (XMC) is a task of finding the most relevant labels from a large label set. Nowadays deep learning-based methods have shown significant success in XMC. However, the existing methods (e.g., AttentionXML and X-Transformer etc) still suffer from 1) combining several models to train and predict for one dataset, and 2) sampling negative labels statically during the process of training label ranking model, which reduces both the efficiency and accuracy of the model. To address the above problems, we proposed LightXML, which adopts end-to-end training and dynamic negative labels sampling. In LightXML, we use generative cooperative networks to recall and rank labels, in which label recalling part generates negative and positive labels, and label ranking part distinguishes positive labels from these labels. Through these networks, negative labels are sampled dynamically during label ranking part training by feeding with the same text representation. Extensive experiments show that LightXML outperforms state-of-the-art methods in five extreme multi-label datasets with much smaller model size and lower computational complexity. In particular, on the Amazon dataset with 670K labels, LightXML can reduce the model size up to 72

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/24/2019

HAXMLNet: Hierarchical Attention Network for Extreme Multi-Label Text Classification

Extreme multi-label text classification (XMTC) addresses the problem of ...
research
10/26/2022

OTSeq2Set: An Optimal Transport Enhanced Sequence-to-Set Model for Extreme Multi-label Text Classification

Extreme multi-label text classification (XMTC) is the task of finding th...
research
12/12/2022

Automated ICD Coding using Extreme Multi-label Long Text Transformer-based Models

Background: Encouraged by the success of pretrained Transformer models i...
research
04/20/2023

Light-weight Deep Extreme Multilabel Classification

Extreme multi-label (XML) classification refers to the task of supervise...
research
10/29/2022

CascadeXML: Rethinking Transformers for End-to-end Multi-resolution Training in Extreme Multi-label Classification

Extreme Multi-label Text Classification (XMC) involves learning a classi...
research
04/11/2019

Ranking-Based Autoencoder for Extreme Multi-label Classification

Extreme Multi-label classification (XML) is an important yet challenging...
research
05/12/2022

Open Vocabulary Extreme Classification Using Generative Models

The extreme multi-label classification (XMC) task aims at tagging conten...

Please sign up or login with your details

Forgot password? Click here to reset