Log In Sign Up

Breaking Down Multilingual Machine Translation

by   Ting-Rui Chiang, et al.

While multilingual training is now an essential ingredient in machine translation (MT) systems, recent work has demonstrated that it has different effects in different multilingual settings, such as many-to-one, one-to-many, and many-to-many learning. These training settings expose the encoder and the decoder in a machine translation model with different data distributions. In this paper, we examine how different varieties of multilingual training contribute to learning these two components of the MT model. Specifically, we compare bilingual models with encoders and/or decoders initialized by multilingual training. We show that multilingual training is beneficial to encoders in general, while it only benefits decoders for low-resource languages (LRLs). We further find the important attention heads for each language pair and compare their correlations during inference. Our analysis sheds light on how multilingual translation models work and also enables us to propose methods to improve performance by training with highly related languages. Our many-to-one models for high-resource languages and one-to-many models for LRL outperform the best results reported by Aharoni et al. (2019).


page 13

page 15


HLT-MT: High-resource Language-specific Training for Multilingual Neural Machine Translation

Multilingual neural machine translation (MNMT) trained in multiple langu...

University of Cape Town's WMT22 System: Multilingual Machine Translation for Southern African Languages

The paper describes the University of Cape Town's submission to the cons...

Competence-based Curriculum Learning for Multilingual Machine Translation

Currently, multilingual machine translation is receiving more and more a...

Building Machine Translation Systems for the Next Thousand Languages

In this paper we share findings from our effort to build practical machi...

Multilingual CheckList: Generation and Evaluation

The recently proposed CheckList (Riberio et al,. 2020) approach to evalu...

A Survey of Orthographic Information in Machine Translation

Machine translation is one of the applications of natural language proce...