DeepAI AI Chat
Log In Sign Up

Addressing the Vulnerability of NMT in Input Perturbations

by   Weiwen Xu, et al.

Neural Machine Translation (NMT) has achieved significant breakthrough in performance but is known to suffer vulnerability to input perturbations. As real input noise is difficult to predict during training, robustness is a big issue for system deployment. In this paper, we improve the robustness of NMT models by reducing the effect of noisy words through a Context-Enhanced Reconstruction (CER) approach. CER trains the model to resist noise in two steps: (1) perturbation step that breaks the naturalness of input sequence with made-up words; (2) reconstruction step that defends the noise propagation by generating better and more robust contextual representation. Experimental results on Chinese-English (ZH-EN) and French-English (FR-EN) translation tasks demonstrate robustness improvement on both news and social media text. Further fine-tuning experiments on social media text show our approach can converge at a higher position and provide a better adaptation.


page 1

page 2

page 3

page 4


Towards Robust Neural Machine Translation

Small perturbations in the input can severely distort intermediate repre...

Robust Neural Machine Translation with Doubly Adversarial Inputs

Neural machine translation (NMT) often suffers from the vulnerability to...

Self-supervised and Supervised Joint Training for Resource-rich Machine Translation

Self-supervised pre-training of text representations has been successful...

Evaluating Robustness to Input Perturbations for Neural Machine Translation

Neural Machine Translation (NMT) models are sensitive to small perturbat...

Ordering Matters: Word Ordering Aware Unsupervised NMT

Denoising-based Unsupervised Neural Machine Translation (U-NMT) models t...

PheMT: A Phenomenon-wise Dataset for Machine Translation Robustness on User-Generated Contents

Neural Machine Translation (NMT) has shown drastic improvement in its qu...

Sometimes We Want Translationese

Rapid progress in Neural Machine Translation (NMT) systems over the last...