DeepAI
Log In Sign Up

HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation

07/11/2022
by   Jian Yang, et al.
0

Multilingual neural machine translation (MNMT) trained in multiple language pairs has attracted considerable attention due to fewer model parameters and lower training costs by sharing knowledge among multiple languages. Nonetheless, multilingual training is plagued by language interference degeneration in shared parameters because of the negative interference among different translation directions, especially on high-resource languages. In this paper, we propose the multilingual translation model with the high-resource language-specific training (HLT-MT) to alleviate the negative interference, which adopts the two-stage training with the language-specific selection mechanism. Specifically, we first train the multilingual model only with the high-resource pairs and select the language-specific modules at the top of the decoder to enhance the translation quality of high-resource directions. Next, the model is further trained on all available corpora to transfer knowledge from high-resource languages (HRLs) to low-resource languages (LRLs). Experimental results show that HLT-MT outperforms various strong baselines on WMT-10 and OPUS-100 benchmarks. Furthermore, the analytic experiments validate the effectiveness of our method in mitigating the negative interference in multilingual training.

READ FULL TEXT

page 2

page 6

10/21/2022

University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languages

The paper describes the University of Cape Town's submission to the cons...
10/15/2021

Breaking Down Multilingual Machine Translation

While multilingual training is now an essential ingredient in machine tr...
04/15/2021

Adaptive Sparse Transformer for Multilingual Translation

Multilingual machine translation has attracted much attention recently d...
10/06/2020

On Negative Interference in Multilingual Models: Findings and A Meta-Learning Treatment

Modern multilingual models are trained on concatenated text from multipl...
07/14/2021

Importance-based Neuron Allocation for Multilingual Neural Machine Translation

Multilingual neural machine translation with a single model has drawn mu...
12/27/2021

Parameter Differentiation based Multilingual Neural Machine Translation

Multilingual neural machine translation (MNMT) aims to translate multipl...
07/20/2021

More Parameters? No Thanks!

This work studies the long-standing problems of model capacity and negat...

Code Repositories

HLT-MT

[IJCAI-ECAI-2022] High-resource Language-specific Training for Multilingual Neural Machine Translation


view repo