Multi-Grained-Attention Gated Convolutional Neural Networks for Sentence Classification

08/22/2018
by   Yang Liu, et al.
0

The classification task of sentences is very challenging because of the limited contextual information they contain. Current methods based on Convolutional Neural Networks (CNNs) for sentence classification do not have a simple and effective mechanism to determine which features in the network are critical and could become large enough that affect the classification result. In this paper, we propose a new CNN model named Multi-Grained-Attention Gated CNN (MGAGCNN), which generates attention weights from the context window of different sizes based on the Multi-Grained-Attention (MGA) gating mechanism. Through the experiments of comparison and attention visualization, we demonstrate that with the help of MGA gating mechanism, our model could learn more meaningful and comprehensive abstract features and adequately identify the critical features, and enhance their influence on the prediction of sentence category. In addition, we propose an activation function named Natural Logarithm rescaled Rectified Linear Unit (NLReLU). Experimental results show that our model could achieve a general accuracy improvement ranging from 0.4 to 1.7 the strong baseline methods on four out of the six tasks, and NLReLU could achieve comparable performances on MGAGCNN to other well-known activation functions.

READ FULL TEXT

page 4

page 5

page 11

page 12

research
08/22/2018

An Attention-Gated Convolutional Neural Network for Sentence Classification

The classification task of sentences is very challenging because of the ...
research
04/08/2018

Comparison of non-linear activation functions for deep neural networks on MNIST classification task

Activation functions play a key role in neural networks so it becomes fu...
research
07/05/2017

Multiple Range-Restricted Bidirectional Gated Recurrent Units with Attention for Relation Classification

Most of neural approaches to relation classification have focused on fin...
research
08/17/2019

A Sensitivity Analysis of Attention-Gated Convolutional Neural Networks for Sentence Classification

Recently, Attention-Gated Convolutional Neural Networks (AGCNNs) perform...
research
07/12/2019

GRN: Gated Relation Network to Enhance Convolutional Neural Network for Named Entity Recognition

The dominant approaches for named entity recognition (NER) mostly adopt ...
research
01/20/2019

Hierarchical Attentional Hybrid Neural Networks for Document Classification

Document classification is a challenging task with important application...
research
06/21/2020

Sequential Feature Filtering Classifier

We propose Sequential Feature Filtering Classifier (FFC), a simple but e...

Please sign up or login with your details

Forgot password? Click here to reset