Pivot-based Transfer Learning for Neural Machine Translation between Non-English Languages

09/20/2019
by   Yunsu Kim, et al.
0

We present effective pre-training strategies for neural machine translation (NMT) using parallel corpora involving a pivot language, i.e., source-pivot and pivot-target, leading to a significant improvement in source-target translation. We propose three methods to increase the relation among source, pivot, and target languages in the pre-training: 1) step-wise training of a single model for different language pairs, 2) additional adapter component to smoothly connect pre-trained encoder and decoder, and 3) cross-lingual encoder training via autoencoding of the pivot language. Our methods greatly outperform multilingual models up to +2.6 tasks. We show that our improvements are valid also in zero-shot/zero-resource scenarios.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/03/2019

Cross-lingual Pre-training Based Transfer for Zero-shot Neural Machine Translation

Transfer learning between different language pairs has shown its effecti...
research
11/15/2016

Neural Machine Translation with Pivot Languages

While recent neural machine translation approaches have delivered state-...
research
11/01/2018

Addressing word-order Divergence in Multilingual Neural Machine Translation for extremely Low Resource Languages

Transfer learning approaches for Neural Machine Translation (NMT) train ...
research
09/17/2020

Code-switching pre-training for neural machine translation

This paper proposes a new pre-training method, called Code-Switching Pre...
research
12/19/2022

Synthetic Pre-Training Tasks for Neural Machine Translation

Pre-training is an effective technique for ensuring robust performance o...
research
10/01/2022

MALM: Mixing Augmented Language Modeling for Zero-Shot Machine Translation

Large pre-trained language models have brought remarkable progress in NL...
research
10/18/2021

Multilingual Domain Adaptation for NMT: Decoupling Language and Domain Information with Adapters

Adapter layers are lightweight, learnable units inserted between transfo...

Please sign up or login with your details

Forgot password? Click here to reset