Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information

05/29/2018
by   Seonhoon Kim, et al.
0

Sentence matching is widely used in various natural language tasks such as natural language inference, paraphrase identification, and question answering. For these tasks, understanding logical and semantic relationship between two sentences is required but it is yet challenging. Although attention mechanism is useful to capture the semantic relationship and to properly align the elements of two sentences, previous methods of attention mechanism simply use a summation operation which does not retain original features enough. Inspired by DenseNet, a densely connected convolutional network, we propose a densely-connected co-attentive recurrent neural network, each layer of which uses concatenated information of attentive features as well as hidden features of all the preceding recurrent layers. It enables preserving the original and the co-attentive feature information from the bottommost word embedding layer to the uppermost recurrent layer. To alleviate the problem of an ever-increasing size of feature vectors due to dense concatenation operations, we also propose to use an autoencoder after dense concatenation. We evaluate our proposed architecture on highly competitive benchmark datasets related to sentence matching. Experimental results show that our architecture, which retains recurrent and attentive features, achieves state-of-the-art performances for all the tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/10/2016

Modelling Sentence Pairs with Tree-structured Attentive Encoder

We describe an attentive encoder that combines tree-structured recursive...
research
02/13/2017

Bilateral Multi-Perspective Matching for Natural Language Sentences

Natural language sentence matching is a fundamental technology for a var...
research
06/13/2018

Generating Sentences Using a Dynamic Canvas

We introduce the Attentive Unsupervised Text (W)riter (AUTR), which is a...
research
08/24/2018

ParaNet - Using Dense Blocks for Early Inference

DenseNets have been shown to be a competitive model among recent convolu...
research
04/24/2021

A Multi-Size Neural Network with Attention Mechanism for Answer Selection

Semantic matching is of central significance to the answer selection tas...
research
06/17/2018

Self-Attentive Neural Collaborative Filtering

The dominant, state-of-the-art collaborative filtering (CF) methods toda...
research
04/10/2018

Question Answering over Freebase via Attentive RNN with Similarity Matrix based CNN

With the rapid growth of knowledge bases (KBs), question answering over ...

Please sign up or login with your details

Forgot password? Click here to reset