Refining Source Representations with Relation Networks for Neural Machine Translation

09/12/2017
by   Wen Zhang, et al.
0

Although neural machine translation (NMT) with the encoder-decoder framework has achieved great success in recent times, it still suffers from some drawbacks: RNNs tend to forget old information which is often useful and the encoder only operates through words without considering word relationship. To solve these problems, we introduce a relation networks (RN) into NMT to refine the encoding representations of the source. In our method, the RN first augments the representation of each source word with its neighbors and reasons all the possible pairwise relations between them. Then the source representations and all the relations are fed to the attention module and the decoder together, keeping the main encoder-decoder architecture unchanged. Experiments on two Chinese-to-English data sets in different scales both show that our method can outperform the competitive baselines significantly.

READ FULL TEXT
research
07/18/2017

Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder

Most neural machine translation (NMT) models are based on the sequential...
research
08/17/2019

Hard but Robust, Easy but Sensitive: How Encoder and Decoder Perform in Neural Machine Translation

Neural machine translation (NMT) typically adopts the encoder-decoder fr...
research
12/26/2018

Learning to Refine Source Representations for Neural Machine Translation

Neural machine translation (NMT) models generally adopt an encoder-decod...
research
01/14/2020

Bi-Decoder Augmented Network for Neural Machine Translation

Neural Machine Translation (NMT) has become a popular technology in rece...
research
06/03/2018

Dense Information Flow for Neural Machine Translation

Recently, neural machine translation has achieved remarkable progress by...
research
09/06/2017

Information-Propogation-Enhanced Neural Machine Translation by Relation Model

Even though sequence-to-sequence neural machine translation (NMT) model ...
research
02/25/2022

Screening Gender Transfer in Neural Machine Translation

This paper aims at identifying the information flow in state-of-the-art ...

Please sign up or login with your details

Forgot password? Click here to reset