Agreement-based Joint Training for Bidirectional Attention-based Neural Machine Translation

by   Yong Cheng, et al.

The attentional mechanism has proven to be effective in improving end-to-end neural machine translation. However, due to the intricate structural divergence between natural languages, unidirectional attention-based models might only capture partial aspects of attentional regularities. We propose agreement-based joint training for bidirectional attention-based end-to-end neural machine translation. Instead of training source-to-target and target-to-source translation models independently,our approach encourages the two complementary models to agree on word alignment matrices on the same training data. Experiments on Chinese-English and English-French translation tasks show that agreement-based joint training significantly improves both alignment and translation quality over independent training.


Supervised Attentions for Neural Machine Translation

In this paper, we improve the attention or alignment accuracy of neural ...

Recurrent Neural Machine Translation

The vanilla attention-based neural machine translation has achieved prom...

On The Alignment Problem In Multi-Head Attention-Based Neural Machine Translation

This work investigates the alignment problem in state-of-the-art multi-h...

Towards Bidirectional Hierarchical Representations for Attention-Based Neural Machine Translation

This paper proposes a hierarchical attentional neural translation model ...

End-to-End Lexically Constrained Machine Translation for Morphologically Rich Languages

Lexically constrained machine translation allows the user to manipulate ...

Modeling Voting for System Combination in Machine Translation

System combination is an important technique for combining the hypothese...

Please sign up or login with your details

Forgot password? Click here to reset