DeepAI AI Chat
Log In Sign Up

QAMPARI: : An Open-domain Question Answering Benchmark for Questions with Many Answers from Multiple Paragraphs

05/25/2022
by   Samuel Joseph Amouyal, et al.
Tel Aviv University
4

Existing benchmarks for open-domain question answering (ODQA) typically focus on questions whose answers can be extracted from a single paragraph. By contrast, many natural questions, such as "What players were drafted by the Brooklyn Nets?" have a list of answers. Answering such questions requires retrieving and reading from many passages, in a large corpus. We introduce QAMPARI, an ODQA benchmark, where question answers are lists of entities, spread across many paragraphs. We created QAMPARI by (a) generating questions with multiple answers from Wikipedia's knowledge graph and tables, (b) automatically pairing answers with supporting evidence in Wikipedia paragraphs, and (c) manually paraphrasing questions and validating each answer. We train ODQA models from the retrieve-and-read family and find that QAMPARI is challenging in terms of both passage retrieval and answer generation, reaching an F1 score of 26.6 at best. Our results highlight the need for developing ODQA models that handle a broad range of question types, including single and multi-answer questions.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/08/2023

Top K Relevant Passage Retrieval for Biomedical Question Answering

Question answering is a task that answers factoid questions using a larg...
08/16/2023

Answering Ambiguous Questions with a Database of Questions, Answers, and Revisions

Many open-domain questions are under-specified and thus have multiple po...
10/13/2021

Open-Domain Question-Answering for COVID-19 and Other Emergent Domains

Since late 2019, COVID-19 has quickly emerged as the newest biomedical d...
02/03/2023

LIQUID: A Framework for List Question Answering Dataset Generation

Question answering (QA) models often rely on large-scale training datase...
06/02/2020

Open-Domain Question Answering with Pre-Constructed Question Spaces

Open-domain question answering aims at solving the task of locating the ...
10/15/2021

MixQG: Neural Question Generation with Mixed Answer Types

Asking good questions is an essential ability for both human and machine...
10/28/2021

What makes us curious? analysis of a corpus of open-domain questions

Every day people ask short questions through smart devices or online for...