DREAM: A Challenge Dataset and Models for Dialogue-Based Reading Comprehension

02/01/2019
by   Kai Sun, et al.
0

We present DREAM, the first dialogue-based multiple-choice reading comprehension dataset. Collected from English-as-a-foreign-language examinations designed by human experts to evaluate the comprehension level of Chinese learners of English, our dataset contains 10,197 multiple-choice questions for 6,444 dialogues. In contrast to existing reading comprehension datasets, DREAM is the first to focus on in-depth multi-turn multi-party dialogue understanding. DREAM is likely to present significant challenges for existing reading comprehension systems: 84 of questions require reasoning beyond a single sentence, and 34 also involve commonsense knowledge. We apply several popular neural reading comprehension models that primarily exploit surface information within the text and find them to, at best, just barely outperform a rule-based approach. We next investigate the effects of incorporating dialogue structure and different kinds of general world knowledge into both rule-based and (neural and non-neural) machine learning-based reading comprehension models. Experimental results on the DREAM dataset show the effectiveness of dialogue structure and general world knowledge. DREAM will be available at https://dataset.org/dream/.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/17/2018

A Span-Extraction Dataset for Chinese Machine Reading Comprehension

Machine Reading Comprehension (MRC) has become enormously popular recent...
research
10/26/2016

Broad Context Language Modeling as Reading Comprehension

Progress in text understanding has been driven by large datasets that te...
research
04/15/2018

What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text

Our goal is to answer questions about paragraphs describing processes (e...
research
09/11/2021

Extract, Integrate, Compete: Towards Verification Style Reading Comprehension

In this paper, we present a new verification style reading comprehension...
research
03/01/2021

Cryptonite: A Cryptic Crossword Benchmark for Extreme Ambiguity in Language

Current NLP datasets targeting ambiguity can be solved by a native speak...
research
02/03/2019

Review Conversational Reading Comprehension

Seeking information about products and services is an important activity...
research
11/02/2019

Design and Challenges of Cloze-Style Reading Comprehension Tasks on Multiparty Dialogue

This paper analyzes challenges in cloze-style reading comprehension on m...

Please sign up or login with your details

Forgot password? Click here to reset