Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehension

05/16/2020
by   Hongyu Gong, et al.
0

In this paper, we study machine reading comprehension (MRC) on long texts, where a model takes as inputs a lengthy document and a question and then extracts a text span from the document as an answer. State-of-the-art models tend to use a pretrained transformer model (e.g., BERT) to encode the joint contextual information of document and question. However, these transformer-based models can only take a fixed-length (e.g., 512) text as its input. To deal with even longer text inputs, previous approaches usually chunk them into equally-spaced segments and predict answers based on each segment independently without considering the information from other segments. As a result, they may form segments that fail to cover the correct answer span or retain insufficient contexts around it, which significantly degrades the performance. Moreover, they are less capable of answering questions that need cross-segment information. We propose to let a model learn to chunk in a more flexible way via reinforcement learning: a model can decide the next segment that it wants to process in either direction. We also employ recurrent mechanisms to enable information to flow across segments. Experiments on three MRC datasets – CoQA, QuAC, and TriviaQA – demonstrate the effectiveness of our proposed recurrent chunking mechanisms: we can obtain segments that are more likely to contain complete answers and at the same time provide sufficient contexts around the ground truth answers for better predictions.

READ FULL TEXT
research
05/16/2020

Recurrent Chunking Mechanisms for Long-Text Machine Reading Comprehensio

In this paper, we study machine reading comprehension (MRC) on long text...
research
05/08/2021

NLP-IIS@UT at SemEval-2021 Task 4: Machine Reading Comprehension using the Long Document Transformer

This paper presents a technical report of our submission to the 4th task...
research
09/29/2019

Tag-based Multi-Span Extraction in Reading Comprehension

With models reaching human performance on many popular reading comprehen...
research
11/04/2016

Learning Recurrent Span Representations for Extractive Question Answering

The reading comprehension task, that asks questions about a given eviden...
research
09/14/2020

Composing Answer from Multi-spans for Reading Comprehension

This paper presents a novel method to generate answers for non-extractio...
research
05/07/2021

VAULT: VAriable Unified Long Text Representation for Machine Reading Comprehension

Existing models on Machine Reading Comprehension (MRC) require complex m...
research
07/06/2023

KoRC: Knowledge oriented Reading Comprehension Benchmark for Deep Text Understanding

Deep text understanding, which requires the connections between a given ...

Please sign up or login with your details

Forgot password? Click here to reset