Reasoning about Entailment with Neural Attention

09/22/2015
by   Tim Rocktäschel, et al.
0

While most approaches to automatically recognizing entailment relations have used classifiers employing hand engineered features derived from complex natural language processing pipelines, in practice their performance has been only slightly better than bag-of-word pair classifiers using only lexical similarity. The only attempt so far to build an end-to-end differentiable neural network for entailment failed to outperform such a simple similarity classifier. In this paper, we propose a neural model that reads two sentences to determine entailment using long short-term memory units. We extend this model with a word-by-word neural attention mechanism that encourages reasoning over entailments of pairs of words and phrases. Furthermore, we present a qualitative analysis of attention weights produced by this model, demonstrating such reasoning capabilities. On a large entailment dataset this model outperforms the previous best neural model and a classifier with engineered features by a substantial margin. It is the first generic end-to-end differentiable system that achieves state-of-the-art accuracy on a textual entailment dataset.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/02/2014

Tiered Clustering to Improve Lexical Entailment

Many tasks in Natural Language Processing involve recognizing lexical en...
research
01/03/2021

Attentive Tree-structured Network for Monotonicity Reasoning

Many state-of-art neural models designed for monotonicity reasoning perf...
research
08/10/2022

Paraphrasing, textual entailment, and semantic similarity above word level

This dissertation explores the linguistic and computational aspects of t...
research
08/13/2018

REGMAPR - A Recipe for Textual Matching

We propose a simple model for textual matching problems. Starting from a...
research
08/13/2018

REGMAPR - Text Matching Made Easy

We propose a simple model for textual matching problems. Starting from a...
research
01/04/2017

Textual Entailment with Structured Attentions and Composition

Deep learning techniques are increasingly popular in the textual entailm...
research
05/25/2017

Max-Cosine Matching Based Neural Models for Recognizing Textual Entailment

Recognizing textual entailment is a fundamental task in a variety of tex...

Please sign up or login with your details

Forgot password? Click here to reset