Densely Connected Attention Propagation for Reading Comprehension

11/10/2018
by   Yi Tay, et al.
0

We propose DecaProp (Densely Connected Attention Propagation), a new densely connected neural architecture for reading comprehension (RC). There are two distinct characteristics of our model. Firstly, our model densely connects all pairwise layers of the network, modeling relationships between passage and query across all hierarchical levels. Secondly, the dense connectors in our network are learned via attention instead of standard residual skip-connectors. To this end, we propose novel Bidirectional Attention Connectors (BAC) for efficiently forging connections throughout the network. We conduct extensive experiments on four challenging RC benchmarks. Our proposed approach achieves state-of-the-art results on all four, outperforming existing baselines by up to 2.6%-14.2% in absolute F1 score.

READ FULL TEXT
research
07/08/2016

Consensus Attention-based Neural Networks for Chinese Reading Comprehension

Reading comprehension has embraced a booming in recent NLP research. Sev...
research
08/05/2020

Densely Connected Residual Network for Attack Recognition

High false alarm rate and low detection rate are the major sticking poin...
research
07/12/2016

Separating Answers from Queries for Neural Reading Comprehension

We present a novel neural architecture for answering queries, designed t...
research
09/29/2020

Lip-reading with Densely Connected Temporal Convolutional Networks

In this work, we present the Densely Connected Temporal Convolutional Ne...
research
08/10/2018

Hierarchical Attention: What Really Counts in Various NLP Tasks

Attention mechanisms in sequence to sequence models have shown great abi...
research
10/19/2018

Lightweight Convolutional Approaches to Reading Comprehension on SQuAD

Current state-of-the-art reading comprehension models rely heavily on re...
research
06/07/2016

Iterative Alternating Neural Attention for Machine Reading

We propose a novel neural attention architecture to tackle machine compr...

Please sign up or login with your details

Forgot password? Click here to reset