Modeling Future Cost for Neural Machine Translation

02/28/2020
by   Chaoqun Duan, et al.
0

Existing neural machine translation (NMT) systems utilize sequence-to-sequence neural networks to generate target translation word by word, and then make the generated word at each time-step and the counterpart in the references as consistent as possible. However, the trained translation model tends to focus on ensuring the accuracy of the generated target word at the current time-step and does not consider its future cost which means the expected cost of generating the subsequent target translation (i.e., the next target word). To respond to this issue, we propose a simple and effective method to model the future cost of each target word for NMT systems. In detail, a time-dependent future cost is estimated based on the current generated target word and its contextual information to boost the training of the NMT model. Furthermore, the learned future context representation at the current time-step is used to help the generation of the next target word in the decoding. Experimental results on three widely-used translation datasets, including the WMT14 German-to-English, WMT14 English-to-French, and WMT17 Chinese-to-English, show that the proposed approach achieves significant improvements over strong Transformer-based NMT baseline.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/08/2020

Explicit Reordering for Neural Machine Translation

In Transformer-based neural machine translation (NMT), the positional en...
research
09/02/2018

Future-Prediction-Based Model for Neural Machine Translation

We propose a novel model for Neural Machine Translation (NMT). Different...
research
06/10/2018

Deconvolution-Based Global Decoding for Neural Machine Translation

A great proportion of sequence-to-sequence (Seq2Seq) models for Neural M...
research
07/19/2017

Modeling Target-Side Inflection in Neural Machine Translation

NMT systems have problems with large vocabulary sizes. Byte-pair encodin...
research
10/09/2019

Novel Applications of Factored Neural Machine Translation

In this work, we explore the usefulness of target factors in neural mach...
research
06/06/2019

Bridging the Gap between Training and Inference for Neural Machine Translation

Neural Machine Translation (NMT) generates target words sequentially in ...
research
10/21/2020

Token Drop mechanism for Neural Machine Translation

Neural machine translation with millions of parameters is vulnerable to ...

Please sign up or login with your details

Forgot password? Click here to reset