Adaptive Nearest Neighbor Machine Translation

05/27/2021
by   Xin Zheng, et al.
0

kNN-MT, recently proposed by Khandelwal et al. (2020a), successfully combines pre-trained neural machine translation (NMT) model with token-level k-nearest-neighbor (kNN) retrieval to improve the translation accuracy. However, the traditional kNN algorithm used in kNN-MT simply retrieves a same number of nearest neighbors for each target token, which may cause prediction errors when the retrieved neighbors include noises. In this paper, we propose Adaptive kNN-MT to dynamically determine the number of k for each target token. We achieve this by introducing a light-weight Meta-k Network, which can be efficiently trained with only a few training samples. On four benchmark machine translation datasets, we demonstrate that the proposed method is able to effectively filter out the noises in retrieval results and significantly outperforms the vanilla kNN-MT model. Even more noteworthy is that the Meta-k Network learned on one domain could be directly applied to other domains and obtain consistent improvements, illustrating the generality of our method. Our implementation is open-sourced at https://github.com/zhengxxn/adaptive-knn-mt.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/23/2023

Simple and Scalable Nearest Neighbor Machine Translation

kNN-MT is a straightforward yet powerful approach for fast domain adapta...
research
05/30/2021

Fast Nearest Neighbor Machine Translation

Though nearest neighbor Machine Translation (kNN-MT) <cit.> has proved t...
research
09/14/2021

Non-Parametric Unsupervised Domain Adaptation for Neural Machine Translation

Recently, kNN-MT has shown the promising capability of directly incorpor...
research
05/26/2023

Bridging the Domain Gaps in Context Representations for k-Nearest Neighbor Neural Machine Translation

k-Nearest neighbor machine translation (kNN-MT) has attracted increasing...
research
10/17/2022

Towards Robust k-Nearest-Neighbor Machine Translation

k-Nearest-Neighbor Machine Translation (kNN-MT) becomes an important res...
research
05/01/2022

Nearest Neighbor Knowledge Distillation for Neural Machine Translation

k-nearest-neighbor machine translation (NN-MT), proposed by Khandelwal e...
research
02/27/2023

kNN-BOX: A Unified Framework for Nearest Neighbor Generation

Augmenting the base neural model with a token-level symbolic datastore i...

Please sign up or login with your details

Forgot password? Click here to reset