An Empirical Study of Generation Order for Machine Translation

by   William Chan, et al.

In this work, we present an empirical study of generation order for machine translation. Building on recent advances in insertion-based modeling, we first introduce a soft order-reward framework that enables us to train models to follow arbitrary oracle generation policies. We then make use of this framework to explore a large variety of generation orders, including uninformed orders, location-based orders, frequency-based orders, content-based orders, and model-based orders. Curiously, we find that for the WMT'14 English → German translation task, order does not have a substantial impact on output quality, with unintuitive orderings such as alphabetical and shortest-first matching the performance of a standard Transformer. This demonstrates that traditional left-to-right generation is not strictly necessary to achieve high performance. On the other hand, results on the WMT'18 English → Chinese task tend to vary more widely, suggesting that translation for less well-aligned language pairs may be more sensitive to generation order.


page 1

page 2

page 3

page 4


Learning and Analyzing Generation Order for Undirected Sequence Models

Undirected neural sequence models have achieved performance competitive ...

Multitask Finetuning for Improving Neural Machine Translation in Indian Languages

Transformer based language models have led to impressive results across ...

Machine Translation between Vietnamese and English: an Empirical Study

Machine translation is shifting to an end-to-end approach based on deep ...

Insertion-based Decoding with Automatically Inferred Generation Order

Conventional neural autoregressive decoding commonly assumes a left-to-r...

Does Order Matter? An Empirical Study on Generating Multiple Keyphrases as a Sequence

Recently, concatenating multiple keyphrases as a target sequence has bee...

Lingua Custodia's participation at the WMT 2021 Machine Translation using Terminologies shared task

This paper describes Lingua Custodia's submission to the WMT21 shared ta...

Dual Reconstruction: a Unifying Objective for Semi-Supervised Neural Machine Translation

While Iterative Back-Translation and Dual Learning effectively incorpora...