BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions

by   Christopher Clark, et al.

In this paper we study yes/no questions that are naturally occurring --- meaning that they are generated in unprompted and unconstrained settings. We build a reading comprehension dataset, BoolQ, of such questions, and show that they are unexpectedly challenging. They often query for complex, non-factoid information, and require difficult entailment-like inference to solve. We also explore the effectiveness of a range of transfer learning baselines. We find that transferring from entailment data is more effective than transferring from paraphrase or extractive QA data, and that it, surprisingly, continues to be very beneficial even when starting from massive pre-trained language models such as BERT. Our best method trains BERT on MultiNLI and then re-trains it on our train set. It achieves 80.4 annotators (and 62 work.



page 1

page 2

page 3

page 4


Transferability of Natural Language Inference to Biomedical Question Answering

Biomedical question answering (QA) is a challenging problem due to the s...

Multilingual Question Answering from Formatted Text applied to Conversational Agents

Recent advances in NLP with language models such as BERT, GPT-2, XLNet o...

ReCAM@IITK at SemEval-2021 Task 4: BERT and ALBERT based Ensemble for Abstract Word Prediction

This paper describes our system for Task 4 of SemEval-2021: Reading Comp...

Neural Skill Transfer from Supervised Language Tasks to Reading Comprehension

Reading comprehension is a challenging task in natural language processi...

How Much Reading Does Reading Comprehension Require? A Critical Investigation of Popular Benchmarks

Many recent papers address reading comprehension, where examples consist...

Knowledge-Empowered Representation Learning for Chinese Medical Reading Comprehension: Task, Model and Resources

Machine Reading Comprehension (MRC) aims to extract answers to questions...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.