Nearest Neighbor Machine Translation

10/01/2020
by   Urvashi Khandelwal, et al.
0

We introduce k-nearest-neighbor machine translation (kNN-MT), which predicts tokens with a nearest neighbor classifier over a large datastore of cached examples, using representations from a neural translation model for similarity search. This approach requires no additional training and scales to give the decoder direct access to billions of examples at test time, resulting in a highly expressive model that consistently improves performance across many settings. Simply adding nearest neighbor search improves a state-of-the-art German-English translation model by 1.5 BLEU. kNN-MT allows a single model to be adapted to diverse domains by using a domain-specific datastore, improving results by an average of 9.2 BLEU over zero-shot transfer, and achieving new state-of-the-art results—without training on these domains. A massively multilingual model can also be specialized for particular language pairs, with improvements of 3 BLEU for translating from English into German and Chinese. Qualitatively, kNN-MT is easily interpretable; it combines source and target context to retrieve highly relevant examples.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/19/2022

Learning Decoupled Retrieval Representation for Nearest Neighbour Neural Machine Translation

K-Nearest Neighbor Neural Machine Translation (kNN-MT) successfully inco...
research
05/23/2023

Non-parametric, Nearest-neighbor-assisted Fine-tuning for Neural Machine Translation

Non-parametric, k-nearest-neighbor algorithms have recently made inroads...
research
10/18/2020

Explaining and Improving Model Behavior with k Nearest Neighbor Representations

Interpretability techniques in NLP have mainly focused on understanding ...
research
06/10/2023

INK: Injecting kNN Knowledge in Nearest Neighbor Machine Translation

Neural machine translation has achieved promising results on many transl...
research
05/30/2021

Fast Nearest Neighbor Machine Translation

Though nearest neighbor Machine Translation (kNN-MT) <cit.> has proved t...
research
03/24/2023

kNN Prompting: Beyond-Context Learning with Calibration-Free Nearest Neighbor Inference

In-Context Learning (ICL), which formulates target tasks as prompt compl...
research
08/11/2022

Domain-Specific Text Generation for Machine Translation

Preservation of domain knowledge from the source to target is crucial in...

Please sign up or login with your details

Forgot password? Click here to reset