Simple and Effective Curriculum Pointer-Generator Networks for Reading Comprehension over Long Narratives

05/26/2019
by   Yi Tay, et al.
8

This paper tackles the problem of reading comprehension over long narratives where documents easily span over thousands of tokens. We propose a curriculum learning (CL) based Pointer-Generator framework for reading/sampling over large documents, enabling diverse training of the neural model based on the notion of alternating contextual difficulty. This can be interpreted as a form of domain randomization and/or generative pretraining during training. To this end, the usage of the Pointer-Generator softens the requirement of having the answer within the context, enabling us to construct diverse training samples for learning. Additionally, we propose a new Introspective Alignment Layer (IAL), which reasons over decomposed alignments using block-based self-attention. We evaluate our proposed method on the NarrativeQA reading comprehension benchmark, achieving state-of-the-art performance, improving existing baselines by 51% relative improvement on BLEU-4 and 17% relative improvement on Rouge-L. Extensive ablations confirm the effectiveness of our proposed IAL and CL components.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/05/2018

Weighted Global Normalization for Multiple Choice ReadingComprehension over Long Documents

Motivated by recent evidence pointing out the fragility of high-performi...
research
10/07/2020

MOCHA: A Dataset for Training and Evaluating Generative Reading Comprehension Metrics

Posing reading comprehension as a generation problem provides a great de...
research
05/16/2018

Joint Training of Candidate Extraction and Answer Selection for Reading Comprehension

While sophisticated neural-based techniques have been developed in readi...
research
03/05/2022

Feeding What You Need by Understanding What You Learned

Machine Reading Comprehension (MRC) reveals the ability to understand a ...
research
09/29/2020

MaP: A Matrix-based Prediction Approach to Improve Span Extraction in Machine Reading Comprehension

Span extraction is an essential problem in machine reading comprehension...
research
01/03/2020

Read Beyond the Lines: Understanding the Implied Textual Meaning via a Skim and Intensive Reading Model

The nonliteral interpretation of a text is hard to be understood by mach...
research
08/28/2019

Discourse-Aware Semantic Self-Attention for Narrative Reading Comprehension

In this work, we propose to use linguistic annotations as a basis for a ...

Please sign up or login with your details

Forgot password? Click here to reset