DeepAI AI Chat
Log In Sign Up

Automatic Task Requirements Writing Evaluation via Machine Reading Comprehension

07/15/2021
by   Shiting Xu, et al.
0

Task requirements (TRs) writing is an important question type in Key English Test and Preliminary English Test. A TR writing question may include multiple requirements and a high-quality essay must respond to each requirement thoroughly and accurately. However, the limited teacher resources prevent students from getting detailed grading instantly. The majority of existing automatic essay scoring systems focus on giving a holistic score but rarely provide reasons to support it. In this paper, we proposed an end-to-end framework based on machine reading comprehension (MRC) to address this problem to some extent. The framework not only detects whether an essay responds to a requirement question, but clearly marks where the essay answers the question. Our framework consists of three modules: question normalization module, ELECTRA based MRC module and response locating module. We extensively explore state-of-the-art MRC methods. Our approach achieves 0.93 accuracy score and 0.85 F1 score on a real-world educational dataset. To encourage reproducible results, we make our code publicly available at <https://github.com/aied2021TRMRC/AIED_2021_TRMRC_code>.

READ FULL TEXT

page 1

page 2

page 3

page 4

02/14/2020

FQuAD: French Question Answering Dataset

Recent advances in the field of language modeling have improved state-of...
04/15/2017

RACE: Large-scale ReAding Comprehension Dataset From Examinations

We present RACE, a new dataset for benchmark evaluation of methods in th...
09/01/2019

Cross-Lingual Machine Reading Comprehension

Though the community has made great progress on Machine Reading Comprehe...
10/12/2018

U-Net: Machine Reading Comprehension with Unanswerable Questions

Machine reading comprehension with unanswerable questions is a new chall...
12/19/2019

An End-to-End Dialogue State Tracking System with Machine Reading Comprehension and Wide Deep Classification

This paper describes our approach in DSTC 8 Track 4: Schema-Guided Dialo...
05/04/2020

To Test Machine Comprehension, Start by Defining Comprehension

Many tasks aim to measure machine reading comprehension (MRC), often foc...
05/13/2022

TIE: Topological Information Enhanced Structural Reading Comprehension on Web Pages

Recently, the structural reading comprehension (SRC) task on web pages h...