DeepAI AI Chat
Log In Sign Up

TellMeWhy: A Dataset for Answering Why-Questions in Narratives

06/11/2021
by   Yash Kumar Lal, et al.
0

Answering questions about why characters perform certain actions is central to understanding and reasoning about narratives. Despite recent progress in QA, it is not clear if existing models have the ability to answer "why" questions that may require commonsense knowledge external to the input narrative. In this work, we introduce TellMeWhy, a new crowd-sourced dataset that consists of more than 30k questions and free-form answers concerning why characters in short narratives perform the actions described. For a third of this dataset, the answers are not present within the narrative. Given the limitations of automated evaluation for this task, we also present a systematized human evaluation interface for this dataset. Our evaluation of state-of-the-art models show that they are far below human performance on answering such questions. They are especially worse on questions whose answers are external to the narrative, thus providing a challenge for future QA and narrative understanding research.

READ FULL TEXT

page 1

page 2

page 3

page 4

10/13/2021

ConditionalQA: A Complex Reading Comprehension Dataset with Conditional Answers

We describe a Question Answering (QA) dataset that contains complex ques...
08/28/2020

A Dataset and Baselines for Visual Question Answering on Art

Answering questions related to art pieces (paintings) is a difficult tas...
09/10/2019

WIQA: A dataset for "What if..." reasoning over procedural text

We introduce WIQA, the first large-scale dataset of "What if..." questio...
05/23/2023

Knowledge of Knowledge: Exploring Known-Unknowns Uncertainty with Large Language Models

This paper investigates the capabilities of Large Language Models (LLMs)...
03/16/2022

Less is More: Summary of Long Instructions is Better for Program Synthesis

Despite the success of large pre-trained language models (LMs) such as C...
04/12/2022

ASQA: Factoid Questions Meet Long-Form Answers

An abundance of datasets and availability of reliable evaluation metrics...
07/10/2017

Automatic Understanding of Image and Video Advertisements

There is more to images than their objective physical content: for examp...