DeepAI
Log In Sign Up

EDITOR: an Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints

11/13/2020
by   Weijia Xu, et al.
0

We introduce an Edit-Based Transformer with Repositioning (EDITOR), which makes sequence generation flexible by seamlessly allowing users to specify preferences in output lexical choice. Building on recent models for non-autoregressive sequence generation (Gu et al., 2019), EDITOR generates new sequences by iteratively editing hypotheses. It relies on a novel reposition operation designed to disentangle lexical choice from word positioning decisions, while enabling efficient oracles for imitation learning and parallel edits at decoding time. Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer (Gu et al., 2019) while speeding up decoding dramatically compared to constrained beam search (Post and Vilar, 2018). EDITOR also achieves comparable or better translation quality with faster decoding speed than the Levenshtein Transformer on standard Romanian-English, English-German, and English-Japanese machine translation tasks.

READ FULL TEXT
04/27/2020

Lexically Constrained Neural Machine Translation with Levenshtein Transformer

This paper proposes a simple and effective algorithm for incorporating l...
06/01/2020

Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English

We conduct in this work an evaluation study comparing offline and online...
03/17/2022

An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models

We propose a framework for training non-autoregressive sequence-to-seque...
08/26/2018

Semi-Autoregressive Neural Machine Translation

Existing approaches to neural machine translation are typically autoregr...
04/24/2017

Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search

We present Grid Beam Search (GBS), an algorithm which extends beam searc...
01/29/2021

Transition based Graph Decoder for Neural Machine Translation

While a number of works showed gains from incorporating source-side symb...
03/17/2021

ENCONTER: Entity Constrained Progressive Sequence Generation via Insertion-based Transformer

Pretrained using large amount of data, autoregressive language models ar...