Nearest Neighbor Machine Translation

10/01/2020
by   Urvashi Khandelwal, et al.
0

We introduce k-nearest-neighbor machine translation (kNN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. kNN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results—without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, kNN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.

READ FULL TEXT

page 1

page 2

page 3

page 4

05/30/2021

Fast Nearest Neighbor Machine Translation

Though nearest neighbor Machine Translation (kNN-MT) <cit.> has proved t...
10/18/2020

Explaining and Improving Model Behavior with k Nearest Neighbor Representations

Interpretability techniques in NLP have mainly focused on understanding ...
10/15/2020

Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings

We describe an unsupervised method to create pseudo-parallel corpora for...
05/01/2022

Nearest Neighbor Knowledge Distillation for Neural Machine Translation

k-nearest-neighbor machine translation (NN-MT), proposed by Khandelwal e...
05/24/2022

Chunk-based Nearest Neighbor Machine Translation

Semi-parametric models, which augment generation with retrieval, have le...
08/11/2022

Domain-Specific Text Generation for Machine Translation

Preservation of domain knowledge from the source to target is crucial in...
12/27/2017

Improving Text Normalization by Optimizing Nearest Neighbor Matching

Text normalization is an essential task in the processing and analysis o...