DeepAI
Log In Sign Up

Narrative Question Answering with Cutting-Edge Open-Domain QA Techniques: A Comprehensive Study

06/07/2021
by   Xiangyang Mou, et al.
0

Recent advancements in open-domain question answering (ODQA), i.e., finding answers from large open-domain corpus like Wikipedia, have led to human-level performance on many datasets. However, progress in QA over book stories (Book QA) lags behind despite its similar task formulation to ODQA. This work provides a comprehensive and quantitative analysis about the difficulty of Book QA: (1) We benchmark the research on the NarrativeQA dataset with extensive experiments with cutting-edge ODQA techniques. This quantifies the challenges Book QA poses, as well as advances the published state-of-the-art with a ∼7% absolute improvement on Rouge-L. (2) We further analyze the detailed challenges in Book QA through human studies.[<https://github.com/gorov/BookQA>.] Our findings indicate that the event-centric questions dominate this task, which exemplifies the inability of existing QA models to handle event-oriented scenarios.

READ FULL TEXT

page 1

page 9

07/20/2020

Frustratingly Hard Evidence Retrieval for QA Over Books

A lot of progress has been made to improve question answering (QA) in re...
11/30/2022

CREPE: Open-Domain Question Answering with False Presuppositions

Information seeking users often pose questions with false presupposition...
04/15/2021

Designing a Minimal Retrieve-and-Read System for Open-Domain Question Answering

In open-domain question answering (QA), retrieve-and-read mechanism has ...
09/08/2018

Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering

We present a new kind of question answering dataset, OpenBookQA, modeled...
08/06/2020

Question and Answer Test-Train Overlap in Open-Domain Question Answering Datasets

Ideally Open-Domain Question Answering models should exhibit a number of...
11/23/2022

Can Open-Domain QA Reader Utilize External Knowledge Efficiently like Humans?

Recent state-of-the-art open-domain QA models are typically based on a t...
10/13/2022

Closed-book Question Generation via Contrastive Learning

Question Generation (QG) is a fundamental NLP task for many downstream a...

Code Repositories