Nearest Neighbor Machine Translation

by   Urvashi Khandelwal, et al.

We introduce k-nearest-neighbor machine translation (kNN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. kNN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results—without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, kNN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.


page 1

page 2

page 3

page 4


Fast Nearest Neighbor Machine Translation

Though nearest neighbor Machine Translation (kNN-MT) <cit.> has proved t...

Explaining and Improving Model Behavior with k Nearest Neighbor Representations

Interpretability techniques in NLP have mainly focused on understanding ...

Unsupervised Bitext Mining and Translation via Self-trained Contextual Embeddings

We describe an unsupervised method to create pseudo-parallel corpora for...

Nearest Neighbor Knowledge Distillation for Neural Machine Translation

k-nearest-neighbor machine translation (NN-MT), proposed by Khandelwal e...

Chunk-based Nearest Neighbor Machine Translation

Semi-parametric models, which augment generation with retrieval, have le...

Domain-Specific Text Generation for Machine Translation

Preservation of domain knowledge from the source to target is crucial in...

Improving Text Normalization by Optimizing Nearest Neighbor Matching

Text normalization is an essential task in the processing and analysis o...