Integrating Weakly Supervised Word Sense Disambiguation into Neural Machine Translation

10/05/2018
by   Xiao Pu, et al.
0

This paper demonstrates that word sense disambiguation (WSD) can improve neural machine translation (NMT) by widening the source context considered when modeling the senses of potentially ambiguous words. We first introduce three adaptive clustering algorithms for WSD, based on k-means, Chinese restaurant processes, and random walks, which are then applied to large word contexts represented in a low-rank space and evaluated on SemEval shared-task data. We then learn word vectors jointly with sense vectors defined by our best WSD method, within a state-of-the-art NMT system. We show that the concatenation of these vectors, and the use of a sense selection mechanism based on the weighted average of sense vectors, outperforms several baselines including sense-aware ones. This is demonstrated by translation on five language pairs. The improvements are above one BLEU point over strong NMT baselines, +4 over all ambiguous nouns and verbs, or +20 challenging words.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/22/2017

Handling Homographs in Neural Machine Translation

Homographs, words with different meanings but the same surface form, hav...
research
08/07/2017

Memory-augmented Neural Machine Translation

Neural machine translation (NMT) has achieved notable success in recent ...
research
06/27/2017

Memory-augmented Chinese-Uyghur Neural Machine Translation

Neural machine translation (NMT) has achieved notable performance recent...
research
05/23/2018

How much does a word weigh? Weighting word embeddings for word sense induction

The paper describes our participation in the first shared task on word s...
research
06/29/2017

Stronger Baselines for Trustable Results in Neural Machine Translation

Interest in neural machine translation has grown rapidly as its effectiv...
research
10/17/2018

An Analysis of Attention Mechanisms: The Case of Word Sense Disambiguation in Neural Machine Translation

Recent work has shown that the encoder-decoder attention mechanisms in n...
research
05/02/2017

Modeling Source Syntax for Neural Machine Translation

Even though a linguistics-free sequence to sequence model in neural mach...

Please sign up or login with your details

Forgot password? Click here to reset