Should we Stop Training More Monolingual Models, and Simply Use Machine Translation Instead?

04/21/2021
by   Tim Isbister, et al.
6

Most work in NLP makes the assumption that it is desirable to develop solutions in the native language in question. There is consequently a strong trend towards building native language models even for low-resource languages. This paper questions this development, and explores the idea of simply translating the data into English, thereby enabling the use of pretrained, and large-scale, English language models. We demonstrate empirically that a large English language model coupled with modern machine translation outperforms native language models in most Scandinavian languages. The exception to this is Finnish, which we assume is due to inferior translation quality. Our results suggest that machine translation is a mature technology, which raises a serious counter-argument for training native language models for low-resource languages. This paper therefore strives to make a provocative but important point. As English language models are improving at an unprecedented pace, which in turn improves machine translation, it is from an empirical and environmental stand-point more effective to translate data from low-resource languages into English, than to build language models for such languages.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/28/2021

A Preordered RNN Layer Boosts Neural Machine Translation in Low Resource Settings

Neural Machine Translation (NMT) models are strong enough to convey sema...
research
02/07/2023

Learning Translation Quality Evaluation on Low Resource Languages from Large Language Models

Learned metrics such as BLEURT have in recent years become widely employ...
research
05/11/2023

How Good are Commercial Large Language Models on African Languages?

Recent advancements in Natural Language Processing (NLP) has led to the ...
research
09/26/2021

Curb Your Carbon Emissions: Benchmarking Carbon Emissions in Machine Translation

In recent times, there has been definitive progress in the field of NLP,...
research
10/11/2022

Enriching Biomedical Knowledge for Low-resource Language Through Translation

Biomedical data and benchmarks are highly valuable yet very limited in l...
research
05/22/2023

GPT-SW3: An Autoregressive Language Model for the Nordic Languages

This paper details the process of developing the first native large gene...
research
01/10/2017

Bidirectional American Sign Language to English Translation

We outline a bidirectional translation system that converts sentences fr...

Please sign up or login with your details

Forgot password? Click here to reset