Reasoning Over Paragraph Effects in Situations

08/16/2019
by   Kevin Lin, et al.
0

A key component of successfully reading a passage of text is the ability to apply knowledge gained from the passage to a new situation. In order to facilitate progress on this kind of reading, we present ROPES, a challenging benchmark for reading comprehension targeting Reasoning Over Paragraph Effects in Situations. We target expository language describing causes and effects (e.g., "animal pollinators increase efficiency of fertilization in flowers"), as they have clear implications for new situations. A system is presented a background passage containing at least one of these relations, a novel situation that uses this background, and questions that require reasoning about effects of the relationships in the background passage in the context of the situation. We collect background passages from science textbooks and Wikipedia that contain such phenomena, and ask crowd workers to author situations, questions, and answers, resulting in a 14,102 question dataset. We analyze the challenges of this task and evaluate the performance of state-of-the-art reading comprehension models. The best model performs only slightly better than randomly guessing an answer of the correct type, at 51.9 human performance of 89.0

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/16/2019

Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning

Machine comprehension of texts longer than a single sentence often requi...
research
11/01/2022

CONDAQA: A Contrastive Reading Comprehension Dataset for Reasoning about Negation

The full power of human language-based communication cannot be realized ...
research
11/13/2020

IIRC: A Dataset of Incomplete Information Reading Comprehension Questions

Humans often have to read multiple documents to address their informatio...
research
10/15/2021

Using Psychological Characteristics of Situations for Social Situation Comprehension in Support Agents

Support agents that help users in their daily lives need to take into ac...
research
10/10/2019

RC-QED: Evaluating Natural Language Derivations in Multi-Hop Reading Comprehension

Recent studies revealed that reading comprehension (RC) systems learn to...
research
04/21/2018

DuoRC: Towards Complex Language Understanding with Paraphrased Reading Comprehension

We propose DuoRC, a novel dataset for Reading Comprehension (RC) that mo...
research
04/15/2018

What Happened? Leveraging VerbNet to Predict the Effects of Actions in Procedural Text

Our goal is to answer questions about paragraphs describing processes (e...

Please sign up or login with your details

Forgot password? Click here to reset