Attention-over-Attention Neural Networks for Reading Comprehension

07/15/2016
by   Yiming Cui, et al.
0

Cloze-style queries are representative problems in reading comprehension. Over the past few months, we have seen much progress that utilizing neural network approach to solve Cloze-style questions. In this paper, we present a novel model called attention-over-attention reader for the Cloze-style reading comprehension task. Our model aims to place another attention mechanism over the document-level attention, and induces "attended attention" for final predictions. Unlike the previous works, our neural network model requires less pre-defined hyper-parameters and uses an elegant architecture for modeling. Experimental results show that the proposed attention-over-attention model significantly outperforms various state-of-the-art systems by a large margin in public datasets, such as CNN and Children's Book Test datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/08/2016

Consensus Attention-based Neural Networks for Chinese Reading Comprehension

Reading comprehension has embraced a booming in recent NLP research. Sev...
research
06/07/2016

Iterative Alternating Neural Attention for Machine Reading

We propose a novel neural attention architecture to tackle machine compr...
research
06/06/2016

Generating and Exploiting Large-scale Pseudo Training Data for Zero Pronoun Resolution

Most existing approaches for zero pronoun resolution are heavily relying...
research
05/05/2017

Sequential Attention: A Context-Aware Alignment Function for Machine Reading

In this paper we propose a neural network model with a novel Sequential ...
research
10/13/2020

Interpreting Attention Models with Human Visual Attention in Machine Reading Comprehension

While neural networks with attention mechanisms have achieved superior p...
research
10/04/2016

Embracing data abundance: BookTest Dataset for Reading Comprehension

There is a practically unlimited amount of natural language data availab...
research
11/23/2016

Emergent Predication Structure in Hidden State Vectors of Neural Readers

A significant number of neural architectures for reading comprehension h...

Please sign up or login with your details

Forgot password? Click here to reset