Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation

10/30/2019
by   Sébastien Jean, et al.
0

Most neural machine translation systems still translate sentences in isolation. To make further progress, a promising line of research additionally considers the surrounding context in order to provide the model potentially missing source-side information, as well as to maintain a coherent output. One difficulty in training such larger-context (i.e. document-level) machine translation systems is that context may be missing from many parallel examples. To circumvent this issue, two-stage approaches, in which sentence-level translations are post-edited in context, have recently been proposed. In this paper, we instead consider the viability of filling in the missing context. In particular, we consider three distinct approaches to generate the missing context: using random contexts, applying a copy heuristic or generating it with a language model. In particular, the copy heuristic significantly helps with lexical coherence, while using completely random contexts hurts performance on many long-distance linguistic phenomena. We also validate the usefulness of tagged back-translation. In addition to improving BLEU scores as expected, using back-translated data helps larger-context machine translation systems to better capture long-range phenomena.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/11/2020

Lexically Cohesive Neural Machine Translation with Copy Mechanism

Lexically cohesive translations preserve consistency in word choices in ...
research
02/20/2021

Understanding and Enhancing the Use of Context for Machine Translation

To understand and infer meaning in language, neural models have to learn...
research
03/12/2019

Context-Aware Learning for Neural Machine Translation

Interest in larger-context neural machine translation, including documen...
research
05/25/2018

Context-Aware Neural Machine Translation Learns Anaphora Resolution

Standard machine translation systems process sentences in isolation and ...
research
06/13/2016

Zero-Resource Translation with Multi-Lingual Neural Machine Translation

In this paper, we propose a novel finetuning algorithm for the recently ...
research
12/06/2018

Context is Key: New Approaches to Neural Coherence Modeling

We formulate coherence modeling as a regression task and propose two nov...
research
01/24/2023

Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction

Crosslingual conditional generation (e.g., machine translation) has long...

Please sign up or login with your details

Forgot password? Click here to reset