MS MARCO: A Human Generated MAchine Reading COmprehension Dataset

by   Tri Nguyen, et al.

This paper presents our recent work on the design and development of a new, large scale dataset, which we name MS MARCO, for MAchine Reading COmprehension.This new dataset is aimed to overcome a number of well-known weaknesses of previous publicly available datasets for the same task of reading comprehension and question answering. In MS MARCO, all questions are sampled from real anonymized user queries. The context passages, from which answers in the dataset are derived, are extracted from real web documents using the most advanced version of the Bing search engine. The answers to the queries are human generated. Finally, a subset of these queries has multiple answers. We aim to release one million queries and the corresponding answers in the dataset, which, to the best of our knowledge, is the most comprehensive real-world dataset of its kind in both quantity and quality. We are currently releasing 100,000 queries with their corresponding answers to inspire work in reading comprehension and question answering along with gathering feedback from the research community.



page 1

page 2

page 3

page 4


PQuAD: A Persian Question Answering Dataset

We present Persian Question Answering Dataset (PQuAD), a crowdsourced re...

Clinical Reading Comprehension: A Thorough Analysis of the emrQA Dataset

Machine reading comprehension has made great progress in recent years ow...

DuReader: a Chinese Machine Reading Comprehension Dataset from Real-world Applications

In this paper, we introduce DuReader, a new large-scale, open-domain Chi...

Adaptations of ROUGE and BLEU to Better Evaluate Machine Reading Comprehension Task

Current evaluation metrics to question answering based machine reading c...

Embracing data abundance: BookTest Dataset for Reading Comprehension

There is a practically unlimited amount of natural language data availab...

CliCR: A Dataset of Clinical Case Reports for Machine Reading Comprehension

We present a new dataset for machine comprehension in the medical domain...

Why can't memory networks read effectively?

Memory networks have been a popular choice among neural architectures fo...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.