The Devil is in the Details: On the Pitfalls of Vocabulary Selection in Neural Machine Translation

05/13/2022
by   Tobias Domhan, et al.
5

Vocabulary selection, or lexical shortlisting, is a well-known technique to improve latency of Neural Machine Translation models by constraining the set of allowed output words during inference. The chosen set is typically determined by separately trained alignment model parameters, independent of the source-sentence context at inference time. While vocabulary selection appears competitive with respect to automatic quality metrics in prior work, we show that it can fail to select the right set of output words, particularly for semantically non-compositional linguistic phenomena such as idiomatic expressions, leading to reduced translation quality as perceived by humans. Trading off latency for quality by increasing the size of the allowed set is often not an option in real-world scenarios. We propose a model of vocabulary selection, integrated into the neural translation model, that predicts the set of allowed output words from contextualized encoder representations. This restores translation quality of an unconstrained system, as measured by human evaluations on WMT newstest2020 and idiomatic expressions, at an inference latency competitive with alignment-based selection using aggressive thresholds, thereby removing the dependency on separately trained alignment models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2016

Vocabulary Manipulation for Neural Machine Translation

In order to capture rich language phenomena, neural machine translation ...
research
12/05/2017

Neural Machine Translation by Generating Multiple Linguistic Factors

Factored neural machine translation (FNMT) is founded on the idea of usi...
research
12/05/2014

On Using Very Large Target Vocabulary for Neural Machine Translation

Neural machine translation, a recently proposed approach to machine tran...
research
05/29/2020

Neural Simultaneous Speech Translation Using Alignment-Based Chunking

In simultaneous machine translation, the objective is to determine when ...
research
10/01/2016

Vocabulary Selection Strategies for Neural Machine Translation

Classical translation models constrain the space of possible outputs by ...
research
10/05/2017

Bilingual Words and Phrase Mappings for Marathi and Hindi SMT

Lack of proper linguistic resources is the major challenges faced by the...
research
02/21/2023

Efficient CTC Regularization via Coarse Labels for End-to-End Speech Translation

For end-to-end speech translation, regularizing the encoder with the Con...

Please sign up or login with your details

Forgot password? Click here to reset