Evaluating Discourse Phenomena in Neural Machine Translation

by   Rachel Bawden, et al.

For machine translation to tackle discourse phenomena, models must have access to extra-sentential linguistic context. There has been recent interest in modelling context in neural machine translation (NMT), but models have been principally evaluated with standard automatic metrics, poorly adapted to evaluating discourse phenomena. In this article, we present hand-crafted, discourse test sets, designed to test the models' ability to exploit previous source and target sentences. We investigate the performance of recently proposed multi-encoder NMT models trained on subtitles for English to French. We also explore a novel way of exploiting context from the previous sentence. Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50 53.5 simple strategy of decoding the concatenation of the previous and current sentence leads to good performance, and our novel strategy of multi-encoding and decoding of two sentences leads to the best performance (72.5 coreference and 57 target-side context.



There are no comments yet.


page 1

page 2

page 3

page 4


Improving Context-aware Neural Machine Translation with Target-side Context

In recent years, several studies on neural machine translation (NMT) hav...

Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns

The advent of context-aware NMT has resulted in promising improvements i...

Exploiting Sentential Context for Neural Machine Translation

In this work, we present novel approaches to exploit sentential context ...

Modeling Coherence for Discourse Neural Machine Translation

Discourse coherence plays an important role in the translation of one te...

Selecting Artificially-Generated Sentences for Fine-Tuning Neural Machine Translation

Neural Machine Translation (NMT) models tend to achieve best performance...

Auto-Encoding Variational Neural Machine Translation

We present a deep generative model of bilingual sentence pairs. The mode...

How Furiously Can Colourless Green Ideas Sleep? Sentence Acceptability in Context

We study the influence of context on sentence acceptability. First we co...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.