XCMRC: Evaluating Cross-lingual Machine Reading Comprehension

08/15/2019
by   Pengyuan Liu, et al.
0

We present XCMRC, the first public cross-lingual language understanding (XLU) benchmark which aims to test machines on their cross-lingual reading comprehension ability. To be specific, XCMRC is a Cross-lingual Cloze-style Machine Reading Comprehension task which requires the reader to fill in a missing word (we additionally provide ten noun candidates) in a sentence written in target language (English / Chinese) by reading a given passage written in source language (Chinese / English). Chinese and English are rich-resource language pairs, in order to study low-resource cross-lingual machine reading comprehension (XMRC), besides defining the common XCMRC task which has no restrictions on use of external language resources, we also define the pseudo low-resource XCMRC task by limiting the language resources to be used. In addition, we provide two baselines for common XCMRC task and two for pseudo XCMRC task respectively. We also provide an upper bound baseline for both tasks. We found that for common XCMRC task, translation-based method and multilingual sentence encoder-based method can obtain reasonable performance but still have much room for improvement. As for pseudo low-resource XCMRC task, due to strict restrictions on the use of language resources, our two approaches are far below the upper bound so there are many challenges ahead.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/01/2019

Cross-Lingual Machine Reading Comprehension

Though the community has made great progress on Machine Reading Comprehe...
research
10/11/2019

BiPaR: A Bilingual Parallel Dataset for Multilingual and Cross-lingual Reading Comprehension on Novels

This paper presents BiPaR, a bilingual parallel novel-style machine read...
research
04/29/2020

Enhancing Answer Boundary Detection for Multilingual Machine Reading Comprehension

Multilingual pre-trained models could leverage the training data from a ...
research
07/11/2021

Improving Low-resource Reading Comprehension via Cross-lingual Transposition Rethinking

Extractive Reading Comprehension (ERC) has made tremendous advances enab...
research
10/27/2020

Cross-lingual Machine Reading Comprehension with Language Branch Knowledge Distillation

Cross-lingual Machine Reading Comprehension (CLMRC) remains a challengin...
research
07/03/2020

Reading Comprehension in Czech via Machine Translation and Cross-lingual Transfer

Reading comprehension is a well studied task, with huge training dataset...
research
06/09/2023

Assisting Language Learners: Automated Trans-Lingual Definition Generation via Contrastive Prompt Learning

The standard definition generation task requires to automatically produc...

Please sign up or login with your details

Forgot password? Click here to reset