Transfer Learning across Languages from Someone Else's NMT Model

09/24/2019
by   Tom Kocmi, et al.
0

Neural machine translation is demanding in terms of training time, hardware resources, size, and quantity of parallel sentences. We propose a simple transfer learning method to recycle already trained models for different language pairs with no need for modifications in model architecture, hyper-parameters, or vocabulary. We achieve better translation quality and shorter convergence times than when training from random initialization. To show the applicability of our method, we recycle a Transformer model trained by different researchers for translating English-to-Czech and used it to seed models for seven language pairs. Our translation models are significantly better even when the re-used model's language pair is not linguistically related to the child language pair, especially for low-resource languages. Our approach needs only one pretrained model for all transferring to all various languages pairs. Additionally, we improve this approach with a simple vocabulary transformation. We analyze the behavior of transfer learning to understand the gains from unrelated languages.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2018

Trivial Transfer Learning for Low-Resource Neural Machine Translation

Transfer learning has been proven as an effective technique for neural m...
research
08/31/2017

Transfer Learning across Low-Resource, Related Languages for Neural Machine Translation

We present a simple method to improve neural translation of a low-resour...
research
11/03/2018

Transfer Learning in Multilingual Neural Machine Translation with Dynamic Vocabulary

We propose a method to transfer knowledge across neural machine translat...
research
01/06/2020

Exploring Benefits of Transfer Learning in Neural Machine Translation

Neural machine translation is known to require large numbers of parallel...
research
09/30/2020

On Romanization for Model Transfer Between Scripts in Neural Machine Translation

Transfer learning is a popular strategy to improve the quality of low-re...
research
09/14/2019

A Universal Parent Model for Low-Resource Neural Machine Translation Transfer

Transfer learning from a high-resource language pair `parent' has been p...
research
04/30/2020

Semi-Supervised Text Simplification with Back-Translation and Asymmetric Denoising Autoencoders

Text simplification (TS) rephrases long sentences into simplified varian...

Please sign up or login with your details

Forgot password? Click here to reset