QA Dataset Explosion: A Taxonomy of NLP Resources for Question Answering and Reading Comprehension

by   Anna Rogers, et al.
Københavns Uni
Allen Institute for Artificial Intelligence

Alongside huge volumes of research on deep learning models in NLP in the recent years, there has been also much work on benchmark datasets needed to track modeling progress. Question answering and reading comprehension have been particularly prolific in this regard, with over 80 new datasets appearing in the past two years. This study is the largest survey of the field to date. We provide an overview of the various formats and domains of the current resources, highlighting the current lacunae for future work. We further discuss the current classifications of “reasoning types" in question answering and propose a new taxonomy. We also discuss the implications of over-focusing on English, and survey the current monolingual resources for other languages and multilingual resources. The study is aimed at both practitioners looking for pointers to the wealth of existing data, and at researchers working on new resources.


page 1

page 2

page 3

page 4


More Than Reading Comprehension: A Survey on Datasets and Metrics of Textual Question Answering

Textual Question Answering (QA) aims to provide precise answers to user'...

FQuAD: French Question Answering Dataset

Recent advances in the field of language modeling have improved state-of...

It Takes Two to Tango: Navigating Conceptualizations of NLP Tasks and Measurements of Performance

Progress in NLP is increasingly measured through benchmarks; hence, cont...

Retrieving and Reading: A Comprehensive Survey on Open-domain Question Answering

Open-domain Question Answering (OpenQA) is an important task in Natural ...

A Framework for Rationale Extraction for Deep QA models

As neural-network-based QA models become deeper and more complex, there ...

Neural Machine Reading Comprehension: Methods and Trends

Machine Reading Comprehension (MRC), which requires the machine to answe...

Please sign up or login with your details

Forgot password? Click here to reset