DeepAI AI Chat
Log In Sign Up

SearchQA: A New Q&A Dataset Augmented with Context from a Search Engine

by   Matthew Dunn, et al.

We publicly release a new large-scale dataset, called SearchQA, for machine comprehension, or question-answering. Unlike recently released datasets, such as DeepMind CNN/DailyMail and SQuAD, the proposed SearchQA was constructed to reflect a full pipeline of general question-answering. That is, we start not from an existing article and generate a question-answer pair, but start from an existing question-answer pair, crawled from J! Archive, and augment it with text snippets retrieved by Google. Following this approach, we built SearchQA, which consists of more than 140k question-answer pairs with each pair having 49.6 snippets on average. Each question-answer-context tuple of the SearchQA comes with additional meta-data such as the snippet's URL, which we believe will be valuable resources for future research. We conduct human evaluation as well as test two baseline methods, one simple word selection and the other deep learning based, on the SearchQA. We show that there is a meaningful gap between the human and machine performances. This suggests that the proposed dataset could well serve as a benchmark for question-answering.


page 1

page 2

page 3

page 4


CodeQA: A Question Answering Dataset for Source Code Comprehension

We propose CodeQA, a free-form question answering dataset for the purpos...

Text Understanding with the Attention Sum Reader Network

Several large cloze-style context-question-answer datasets have been int...

BiRdQA: A Bilingual Dataset for Question Answering on Tricky Riddles

A riddle is a question or statement with double or veiled meanings, foll...

QUADRo: Dataset and Models for QUestion-Answer Database Retrieval

An effective paradigm for building Automated Question Answering systems ...

Context-guided Triple Matching for Multiple Choice Question Answering

The task of multiple choice question answering (MCQA) refers to identify...

Towards Solving Multimodal Comprehension

This paper targets the problem of procedural multimodal machine comprehe...

Code Repositories