Log In Sign Up

Explicit Reordering for Neural Machine Translation

by   Kehai Chen, et al.

In Transformer-based neural machine translation (NMT), the positional encoding mechanism helps the self-attention networks to learn the source representation with order dependency, which makes the Transformer-based NMT achieve state-of-the-art results for various translation tasks. However, Transformer-based NMT only adds representations of positions sequentially to word vectors in the input sentence and does not explicitly consider reordering information in this sentence. In this paper, we first empirically investigate the relationship between source reordering information and translation performance. The empirical findings show that the source input with the target order learned from the bilingual parallel dataset can substantially improve translation performance. Thus, we propose a novel reordering method to explicitly model this reordering information for the Transformer-based NMT. The empirical results on the WMT14 English-to-German, WAT ASPEC Japanese-to-English, and WMT17 Chinese-to-English translation tasks show the effectiveness of the proposed approach.


page 1

page 2

page 3

page 4


Explicit Sentence Compression for Neural Machine Translation

State-of-the-art Transformer-based neural machine translation (NMT) syst...

Improving Neural Machine Translation with Parent-Scaled Self-Attention

Most neural machine translation (NMT) models operate on source and targe...

Prevent the Language Model from being Overconfident in Neural Machine Translation

The Neural Machine Translation (NMT) model is essentially a joint langua...

Lattice-Based Transformer Encoder for Neural Machine Translation

Neural machine translation (NMT) takes deterministic sequences for sourc...

Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation

Traditional Neural machine translation (NMT) involves a fixed training p...

Modeling Future Cost for Neural Machine Translation

Existing neural machine translation (NMT) systems utilize sequence-to-se...

Graph-to-Sequence Neural Machine Translation

Neural machine translation (NMT) usually works in a seq2seq learning way...