AdvAug: Robust Adversarial Augmentation for Neural Machine Translation

06/21/2020
by   Yong Cheng, et al.
0

In this paper, we propose a new adversarial augmentation method for Neural Machine Translation (NMT). The main idea is to minimize the vicinal risk over virtual sentences sampled from two vicinity distributions, of which the crucial one is a novel vicinity distribution for adversarial sentences that describes a smooth interpolated embedding space centered around observed training sentence pairs. We then discuss our approach, AdvAug, to train NMT models using the embeddings of virtual sentences in sequence-to-sequence learning. Experiments on Chinese-English, English-French, and English-German translation benchmarks show that AdvAug achieves significant improvements over the Transformer (up to 4.9 BLEU points), and substantially outperforms other data augmentation techniques (e.g. back-translation) without using extra corpora.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/07/2021

Developing neural machine translation models for Hungarian-English

I train models for the task of neural machine translation for English-Hu...
research
04/24/2018

Unsupervised Neural Machine Translation with Weight Sharing

Unsupervised neural machine translation (NMT) is a recently proposed app...
research
11/11/2019

Diversity by Phonetics and its Application in Neural Machine Translation

We introduce a powerful approach for Neural Machine Translation (NMT), w...
research
03/15/2017

Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets

This paper proposes an approach for applying GANs to NMT. We build a con...
research
10/09/2020

Uncertainty-Aware Semantic Augmentation for Neural Machine Translation

As a sequence-to-sequence generation task, neural machine translation (N...
research
05/01/2018

Dynamic Sentence Sampling for Efficient Training of Neural Machine Translation

Traditional Neural machine translation (NMT) involves a fixed training p...
research
11/05/2019

Training Neural Machine Translation (NMT) Models using Tensor Train Decomposition on TensorFlow (T3F)

We implement a Tensor Train layer in the TensorFlow Neural Machine Trans...

Please sign up or login with your details

Forgot password? Click here to reset