Log In Sign Up

Evaluating Discourse Phenomena in Neural Machine Translation

by   Rachel Bawden, et al.

For machine translation to tackle discourse phenomena, models must have access to extra-sentential linguistic context. There has been recent interest in modelling context in neural machine translation (NMT), but models have been principally evaluated with standard automatic metrics, poorly adapted to evaluating discourse phenomena. In this article, we present hand-crafted, discourse test sets, designed to test the models' ability to exploit previous source and target sentences. We investigate the performance of recently proposed multi-encoder NMT models trained on subtitles for English to French. We also explore a novel way of exploiting context from the previous sentence. Despite gains using BLEU, multi-encoder models give limited improvement in the handling of discourse phenomena: 50 53.5 simple strategy of decoding the concatenation of the previous and current sentence leads to good performance, and our novel strategy of multi-encoding and decoding of two sentences leads to the best performance (72.5 coreference and 57 target-side context.


page 1

page 2

page 3

page 4


Improving Context-aware Neural Machine Translation with Target-side Context

In recent years, several studies on neural machine translation (NMT) hav...

Focused Concatenation for Context-Aware Neural Machine Translation

A straightforward approach to context-aware neural machine translation c...

Contextual Neural Machine Translation Improves Translation of Cataphoric Pronouns

The advent of context-aware NMT has resulted in promising improvements i...

Exploiting Sentential Context for Neural Machine Translation

In this work, we present novel approaches to exploit sentential context ...

Context in Neural Machine Translation: A Review of Models and Evaluations

This review paper discusses how context has been used in neural machine ...

Auto-Encoding Variational Neural Machine Translation

We present a deep generative model of bilingual sentence pairs. The mode...

Improving Topic Segmentation by Injecting Discourse Dependencies

Recent neural supervised topic segmentation models achieve distinguished...