DeepAI AI Chat
Log In Sign Up

Weakly Supervised Grammatical Error Correction using Iterative Decoding

10/31/2018
by   Jared Lichtarge, et al.
0

We describe an approach to Grammatical Error Correction (GEC) that is effective at making use of models trained on large amounts of weakly supervised bitext. We train the Transformer sequence-to-sequence model on 4B tokens of Wikipedia revisions and employ an iterative decoding strategy that is tailored to the loosely-supervised nature of the Wikipedia training corpus. Finetuning on the Lang-8 corpus and ensembling yields an F0.5 of 58.3 on the CoNLL'14 benchmark and a GLEU of 62.4 on JFLEG. The combination of weakly supervised training and iterative decoding obtains an F0.5 of 48.2 on CoNLL'14 even without using any labeled GEC data.

READ FULL TEXT

page 1

page 2

page 3

page 4

04/10/2019

Corpora Generation for Grammatical Error Correction

Grammatical Error Correction (GEC) has been recently modeled using the s...
09/29/2019

Controllable Data Synthesis Method for Grammatical Error Correction

Due to the lack of parallel data in current Grammatical Error Correction...
10/07/2019

Parallel Iterative Edit Models for Local Sequence Transduction

We present a Parallel Iterative Edit (PIE) model for the problem of loca...
05/29/2021

Grammatical Error Correction as GAN-like Sequence Labeling

In Grammatical Error Correction (GEC), sequence labeling models enjoy fa...
11/22/2022

Converge to the Truth: Factual Error Correction via Iterative Constrained Editing

Given a possibly false claim sentence, how can we automatically correct ...
04/20/2017

Hard Mixtures of Experts for Large Scale Weakly Supervised Vision

Training convolutional networks (CNN's) that fit on a single GPU with mi...
10/27/2019

Training ASR models by Generation of Contextual Information

Supervised ASR models have reached unprecedented levels of accuracy, tha...