An Empirical Study of Generation Order for Machine Translation

10/29/2019
by   William Chan, et al.
0

In this work, we present an empirical study of generation order for machine translation. Building on recent advances in insertion-based modeling, we first introduce a soft order-reward framework that enables us to train models to follow arbitrary oracle generation policies. We then make use of this framework to explore a large variety of generation orders, including uninformed orders, location-based orders, frequency-based orders, content-based orders, and model-based orders. Curiously, we find that for the WMT'14 English → German translation task, order does not have a substantial impact on output quality, with unintuitive orderings such as alphabetical and shortest-first matching the performance of a standard Transformer. This demonstrates that traditional left-to-right generation is not strictly necessary to achieve high performance. On the other hand, results on the WMT'18 English → Chinese task tend to vary more widely, suggesting that translation for less well-aligned language pairs may be more sensitive to generation order.

READ FULL TEXT

page 1

page 2

page 3

page 4

12/16/2021

Learning and Analyzing Generation Order for Undirected Sequence Models

Undirected neural sequence models have achieved performance competitive ...
12/03/2021

Multitask Finetuning for Improving Neural Machine Translation in Indian Languages

Transformer based language models have led to impressive results across ...
10/30/2018

Machine Translation between Vietnamese and English: an Empirical Study

Machine translation is shifting to an end-to-end approach based on deep ...
02/04/2019

Insertion-based Decoding with Automatically Inferred Generation Order

Conventional neural autoregressive decoding commonly assumes a left-to-r...
09/09/2019

Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence

Recently, concatenating multiple keyphrases as a target sequence has bee...
11/03/2021

Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task

This paper describes Lingua Custodia's submission to the WMT21 shared ta...
10/07/2020

Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation

While Iterative Back-Translation and Dual Learning effectively incorpora...