Log In Sign Up

Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study

by   Xiangyang Mou, et al.

Recent advancements in open-domain question answering (ODQA), i.e., finding answers from large open-domain corpus like Wikipedia, have led to human-level performance on many datasets. However, progress in QA over book stories (Book QA) lags behind despite its similar task formulation to ODQA. This work provides a comprehensive and quantitative analysis about the difficulty of Book QA: (1) We benchmark the research on the NarrativeQA dataset with extensive experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a ∼7% absolute improvement on Rouge-L. (2) We further analyze the detailed challenges in Book QA through human studies.[<>.] Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.


page 1

page 9


Frustratingly Hard Evidence Retrieval for QA Over Books

A lot of progress has been made to improve question answering (QA) in re...

CREPE: Open-Domain Question Answering with False Presuppositions

Information seeking users often pose questions with false presupposition...

Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering

In open-domain question answering (QA), retrieve-and-read mechanism has ...

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

We present a new kind of question answering dataset, OpenBookQA, modeled...

Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets

Ideally Open-Domain Question Answering models should exhibit a number of...

Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?

Recent state-of-the-art open-domain QA models are typically based on a t...

Closed-book Question Generation via Contrastive Learning

Question Generation (QG) is a fundamental NLP task for many downstream a...

Code Repositories