A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning

07/02/2019
by   Yo Joong Choe, et al.
0

Grammatical error correction can be viewed as a low-resource sequence-to-sequence task, because publicly available parallel corpora are limited. To tackle this challenge, we first generate erroneous versions of large unannotated corpora using a realistic noising function. The resulting parallel corpora are subsequently used to pre-train Transformer models. Then, by sequentially applying transfer learning, we adapt these models to the domain and style of the test set. Combined with a context-aware neural spellchecker, our system achieves competitive results in both restricted and low resource tracks in ACL 2019 BEA Shared Task. We release all of our code and materials for reproducibility.

READ FULL TEXT
research
10/01/2019

Grammatical Error Correction in Low-Resource Scenarios

Grammatical error correction in English is a long studied problem with m...
research
04/16/2018

Approaching Neural Grammatical Error Correction as a Low-Resource Machine Translation Task

Previously, neural methods in grammatical error correction (GEC) did not...
research
09/12/2019

CUNI System for the Building Educational Applications 2019 Shared Task: Grammatical Error Correction

In this paper, we describe our systems submitted to the Building Educati...
research
05/26/2020

GECToR – Grammatical Error Correction: Tag, Not Rewrite

In this paper, we present a simple and efficient GEC sequence tagger usi...
research
05/07/2020

JASS: Japanese-specific Sequence to Sequence Pre-training for Neural Machine Translation

Neural machine translation (NMT) needs large parallel corpora for state-...
research
04/10/2019

Corpora Generation for Grammatical Error Correction

Grammatical Error Correction (GEC) has been recently modeled using the s...
research
03/01/2019

Improving Grammatical Error Correction via Pre-Training a Copy-Augmented Architecture with Unlabeled Data

Neural machine translation systems have become state-of-the-art approach...

Please sign up or login with your details

Forgot password? Click here to reset