RACE: Large-scale ReAding Comprehension Dataset From Examinations

04/15/2017
by   Guokun Lai, et al.
0

We present RACE, a new dataset for benchmark evaluation of methods in the reading comprehension task. Collected from the English exams for middle and high school Chinese students in the age range between 12 to 18, RACE consists of near 28,000 passages and near 100,000 questions generated by human experts (English instructors), and covers a variety of topics which are carefully designed for evaluating the students' ability in understanding and reasoning. In particular, the proportion of questions that requires reasoning is much larger in RACE than that in other benchmark datasets for reading comprehension, and there is a significant gap between the performance of the state-of-the-art models (43 can serve as a valuable resource for research and evaluation in machine comprehension. The dataset is freely available at http://www.cs.cmu.edu/ glai1/data/race/ and the code is available at https://github.com/qizhex/RACE_AR_baselines.

READ FULL TEXT
research
10/17/2018

A Span-Extraction Dataset for Chinese Machine Reading Comprehension

Machine Reading Comprehension (MRC) has become enormously popular recent...
research
04/29/2020

Benchmarking Robustness of Machine Reading Comprehension Models

Machine Reading Comprehension (MRC) is an important testbed for evaluati...
research
02/28/2019

FastFusionNet: New State-of-the-Art for DAWNBench SQuAD

In this technical report, we introduce FastFusionNet, an efficient varia...
research
05/15/2023

EMBRACE: Evaluation and Modifications for Boosting RACE

When training and evaluating machine reading comprehension models, it is...
research
04/04/2019

Frustratingly Poor Performance of Reading Comprehension Models on Non-adversarial Examples

When humans learn to perform a difficult task (say, reading comprehensio...
research
07/15/2021

Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension

Task requirements (TRs) writing is an important question type in Key Eng...
research
06/04/2019

ChID: A Large-scale Chinese IDiom Dataset for Cloze Test

Cloze-style reading comprehension in Chinese is still limited due to the...

Please sign up or login with your details

Forgot password? Click here to reset