Evaluating the Supervised and Zero-shot Performance of Multi-lingual Translation Models

06/24/2019
by   Chris Hokamp, et al.
0

We study several methods for full or partial sharing of the decoder parameters of multilingual NMT models. We evaluate both fully supervised and zero-shot translation performance in 110 unique translation directions using only the WMT 2019 shared task parallel datasets for training. We use additional test sets and re-purpose evaluation methods recently used for unsupervised MT in order to evaluate zero-shot translation performance for language pairs where no gold-standard parallel data is available. To our knowledge, this is the largest evaluation of multi-lingual translation yet conducted in terms of the total size of the training data we use, and in terms of the diversity of zero-shot translation pairs we evaluate. We conduct an in-depth evaluation of the translation performance of different models, highlighting the trade-offs between methods of sharing decoder parameters. We find that models which have task-specific decoder parameters outperform models where decoder parameters are fully shared across all tasks.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/04/2019

Consistency by Agreement in Zero-shot Neural Machine Translation

Generalization and reliability of multilingual translation often highly ...
research
09/12/2018

Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

Transferring representations from large supervised tasks to downstream t...
research
05/25/2018

Zero-Shot Dual Machine Translation

Neural Machine Translation (NMT) systems rely on large amounts of parall...
research
02/22/2022

RuCLIP – new models and experiments: a technical report

In the report we propose six new implementations of ruCLIP model trained...
research
05/26/2023

Gender Lost In Translation: How Bridging The Gap Between Languages Affects Gender Bias in Zero-Shot Multilingual Translation

Neural machine translation (NMT) models often suffer from gender biases ...
research
05/23/2023

Exploring Representational Disparities Between Multilingual and Bilingual Translation Models

Multilingual machine translation has proven immensely useful for low-res...
research
05/26/2023

Automatic Tuning of Loss Trade-offs without Hyper-parameter Search in End-to-End Zero-Shot Speech Synthesis

Recently, zero-shot TTS and VC methods have gained attention due to thei...

Please sign up or login with your details

Forgot password? Click here to reset