Do Grammatical Error Correction Models Realize Grammatical Generalization?

by   Masato Mita, et al.

There has been an increased interest in data generation approaches to grammatical error correction (GEC) using pseudo data. However, these approaches suffer from several issues that make them inconvenient for real-world deployment including a demand for large amounts of training data. On the other hand, some errors based on grammatical rules may not necessarily require a large amount of data if GEC models can realize grammatical generalization. This study explores to what extent GEC models generalize grammatical knowledge required for correcting errors. We introduce an analysis method using synthetic and real GEC datasets with controlled vocabularies to evaluate whether models can generalize to unseen errors. We found that a current standard Transformer-based GEC model fails to realize grammatical generalization even in simple settings with limited vocabulary and syntax, suggesting that it lacks the generalization ability required to correct errors from provided training examples.



There are no comments yet.


page 1

page 2

page 3

page 4


Correcting the Autocorrect: Context-Aware Typographical Error Correction via Training Data Augmentation

In this paper, we explore the artificial generation of typographical err...

Towards Minimal Supervision BERT-based Grammar Error Correction

Current grammatical error correction (GEC) models typically consider the...

Grammatical Error Generation Based on Translated Fragments

We perform neural machine translation of sentence fragments in order to ...

Spelling Error Correction Using a Nested RNN Model and Pseudo Training Data

We propose a nested recurrent neural network (nested RNN) model for Engl...

Information Spread with Error Correction

We study the process of information dispersal in a network with communic...

A Syntax-Guided Grammatical Error Correction Model with Dependency Tree Correction

Grammatical Error Correction (GEC) is a task of detecting and correcting...

MILR: Mathematically Induced Layer Recovery for Plaintext Space Error Correction of CNNs

The increased use of Convolutional Neural Networks (CNN) in mission crit...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.