Does Multi-Encoder Help? A Case Study on Context-Aware Neural Machine Translation

05/07/2020
by   Bei Li, et al.
0

In encoder-decoder neural models, multiple encoders are in general used to represent the contextual information in addition to the individual sentence. In this paper, we investigate multi-encoder approaches in documentlevel neural machine translation (NMT). Surprisingly, we find that the context encoder does not only encode the surrounding sentences but also behaves as a noise generator. This makes us rethink the real benefits of multi-encoder in context-aware translation - some of the improvements come from robust training. We compare several methods that introduce noise and/or well-tuned dropout setup into the training of these encoders. Experimental results show that noisy training plays an important role in multi-encoder-based NMT, especially when the training data is small. Also, we establish a new state-of-the-art on IWSLT Fr-En task by careful use of noise generation and dropout methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2019

Improving Context-aware Neural Machine Translation with Target-side Context

In recent years, several studies on neural machine translation (NMT) hav...
research
04/14/2017

Exploiting Cross-Sentence Context for Neural Machine Translation

In translation, considering the document as a whole can help to resolve ...
research
09/05/2018

Document-Level Neural Machine Translation with Hierarchical Attention Networks

Neural Machine Translation (NMT) can be improved by including document-l...
research
03/31/2021

Divide and Rule: Training Context-Aware Multi-Encoder Translation Models with Little Resources

Multi-encoder models are a broad family of context-aware Neural Machine ...
research
08/18/2023

Is context all you need? Scaling Neural Sign Language Translation to Large Domains of Discourse

Sign Language Translation (SLT) is a challenging task that aims to gener...
research
02/28/2022

LCP-dropout: Compression-based Multiple Subword Segmentation for Neural Machine Translation

In this study, we propose a simple and effective preprocessing method fo...
research
08/30/2019

Encoders Help You Disambiguate Word Senses in Neural Machine Translation

Neural machine translation (NMT) has achieved new state-of-the-art perfo...

Please sign up or login with your details

Forgot password? Click here to reset