Log In Sign Up

EDITOR: an Edit-Based Transformer with Repositioning for Neural Machine Translation with Soft Lexical Constraints

by   Weijia Xu, et al.

We introduce an Edit-Based Transformer with Repositioning (EDITOR), which makes sequence generation flexible by seamlessly allowing users to specify preferences in output lexical choice. Building on recent models for non-autoregressive sequence generation (Gu et al., 2019), EDITOR generates new sequences by iteratively editing hypotheses. It relies on a novel reposition operation designed to disentangle lexical choice from word positioning decisions, while enabling efficient oracles for imitation learning and parallel edits at decoding time. Empirically, EDITOR uses soft lexical constraints more effectively than the Levenshtein Transformer (Gu et al., 2019) while speeding up decoding dramatically compared to constrained beam search (Post and Vilar, 2018). EDITOR also achieves comparable or better translation quality with faster decoding speed than the Levenshtein Transformer on standard Romanian-English, English-German, and English-Japanese machine translation tasks.


Lexically Constrained Neural Machine Translation with Levenshtein Transformer

This paper proposes a simple and effective algorithm for incorporating l...

Online Versus Offline NMT Quality: An In-depth Analysis on English-German and German-English

We conduct in this work an evaluation study comparing offline and online...

An Imitation Learning Curriculum for Text Editing with Non-Autoregressive Models

We propose a framework for training non-autoregressive sequence-to-seque...

Semi-Autoregressive Neural Machine Translation

Existing approaches to neural machine translation are typically autoregr...

Lexically Constrained Decoding for Sequence Generation Using Grid Beam Search

We present Grid Beam Search (GBS), an algorithm which extends beam searc...

Transition based Graph Decoder for Neural Machine Translation

While a number of works showed gains from incorporating source-side symb...

ENCONTER: Entity Constrained Progressive Sequence Generation via Insertion-based Transformer

Pretrained using large amount of data, autoregressive language models ar...