Meta-Curriculum Learning for Domain Adaptation in Neural Machine Translation

03/03/2021
by   Runzhe Zhan, et al.
0

Meta-learning has been sufficiently validated to be beneficial for low-resource neural machine translation (NMT). However, we find that meta-trained NMT fails to improve the translation performance of the domain unseen at the meta-training stage. In this paper, we aim to alleviate this issue by proposing a novel meta-curriculum learning for domain adaptation in NMT. During meta-training, the NMT first learns the similar curricula from each domain to avoid falling into a bad local optimum early, and finally learns the curricula of individualities to improve the model robustness for learning domain-specific knowledge. Experimental results on 10 different low-resource domains show that meta-curriculum learning can improve the translation performance of both familiar and unfamiliar domains. All the codes and data are freely available at https://github.com/NLP2CT/Meta-Curriculum.

READ FULL TEXT

page 1

page 2

page 3

page 4

08/26/2019

An Empirical Study of Domain Adaptation for Unsupervised Neural Machine Translation

Domain adaptation methods have been well-studied in supervised neural ma...
04/06/2020

Meta-Learning for Few-Shot NMT Adaptation

We present META-MT, a meta-learning approach to adapt Neural Machine Tra...
12/15/2021

Improving both domain robustness and domain adaptability in machine translation

We address two problems of domain adaptation in neural machine translati...
10/18/2020

Meta-Learning for Low-Resource Unsupervised Neural MachineTranslation

Unsupervised machine translation, which utilizes unpaired monolingual co...
03/20/2021

Token-wise Curriculum Learning for Neural Machine Translation

Existing curriculum learning approaches to Neural Machine Translation (N...
11/30/2020

Dynamic Curriculum Learning for Low-Resource Neural Machine Translation

Large amounts of data has made neural machine translation (NMT) a big su...
04/07/2020

Self-Induced Curriculum Learning in Neural Machine Translation

Self-supervised neural machine translation (SS-NMT) learns how to extrac...