Incorporating Effective Global Information via Adaptive Gate Attention for Text Classification

02/22/2020
by   Xianming Li, et al.
0

The dominant text classification studies focus on training classifiers using textual instances only or introducing external knowledge (e.g., hand-craft features and domain expert knowledge). In contrast, some corpus-level statistical features, like word frequency and distribution, are not well exploited. Our work shows that such simple statistical information can enhance classification performance both efficiently and significantly compared with several baseline models. In this paper, we propose a classifier with gate mechanism named Adaptive Gate Attention model with Global Information (AGA+GI), in which the adaptive gate mechanism incorporates global statistical features into latent semantic features and the attention layer captures dependency relationship within the sentence. To alleviate the overfitting issue, we propose a novel Leaky Dropout mechanism to improve generalization ability and performance stability. Our experiments show that the proposed method can achieve better accuracy than CNN-based and RNN-based approaches without global information on several benchmarks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/24/2018

From Random to Supervised: A Novel Dropout Mechanism Integrated with Global Information

Dropout is used to avoid overfitting by randomly dropping units from the...
research
09/10/2022

Adaptive Meta-learner via Gradient Similarity for Few-shot Text Classification

Few-shot text classification aims to classify the text under the few-sho...
research
08/26/2018

Semantic-Unit-Based Dilated Convolution for Multi-Label Text Classification

We propose a novel model for multi-label text classification, which is b...
research
04/21/2019

3G structure for image caption generation

It is a big challenge of computer vision to make machine automatically d...
research
08/04/2021

TextCNN with Attention for Text Classification

The vast majority of textual content is unstructured, making automated c...
research
02/01/2019

tax2vec: Constructing Interpretable Features from Taxonomies for Short Text Classification

The use of background knowledge remains largely unexploited in many text...
research
02/21/2019

Deep Short Text Classification with Knowledge Powered Attention

Short text classification is one of important tasks in Natural Language ...

Please sign up or login with your details

Forgot password? Click here to reset