b'Orhan Firat'

research

∙ 05/19/2023

Cross-Lingual Supervision improves Large Language Models Pre-training

The recent rapid progress in pre-training Large Language Models has reli...

0 Andrea Schioppa, et al. ∙

research

∙ 04/18/2023

UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining

Pretrained multilingual large language models have typically used heuris...

0 Hyung Won Chung, et al. ∙

research

∙ 03/27/2023

Bilex Rx: Lexical Data Augmentation for Massively Multilingual Machine Translation

Neural machine translation (NMT) has progressed rapidly over the past se...

0 Alex Jones, et al. ∙

research

∙ 02/19/2023

Scaling Laws for Multilingual Neural Machine Translation

In this work, we provide a large-scale empirical study of the scaling pr...

0 Patrick Fernandes, et al. ∙

research

∙ 02/09/2023

Binarized Neural Machine Translation

The rapid scaling of language models is motivating research using low-bi...

0 Yichi Zhang, et al. ∙

research

∙ 02/02/2023

The unreasonable effectiveness of few-shot learning for machine translation

We demonstrate the potential of few-shot translation systems, trained wi...

0 Xavier Garcia, et al. ∙

research

∙ 01/24/2023

Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction

Crosslingual conditional generation (e.g., machine translation) has long...

0 Jonathan Pilault, et al. ∙

research

∙ 10/01/2022

FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation

We present FRMT, a new dataset and evaluation benchmark for Few-shot Reg...

0 Parker Riley, et al. ∙

research

∙ 09/23/2022

Do Current Multi-Task Optimization Methods in Deep Learning Even Help?

Recent research has proposed a series of specialized optimization algori...

36 Derrick Xin, et al. ∙

research

∙ 05/09/2022

Building Machine Translation Systems for the Next Thousand Languages

In this paper we share findings from our effort to build practical machi...

8 Ankur Bapna, et al. ∙

research

∙ 04/05/2022

PaLM: Scaling Language Modeling with Pathways

Large language models have been shown to achieve remarkable performance ...

6 Aakanksha Chowdhery, et al. ∙

research

∙ 03/21/2022

XTREME-S: Evaluating Cross-lingual Speech Representations

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingu...

0 Alexis Conneau, et al. ∙

research

∙ 03/15/2022

Multilingual Mix: Example Interpolation Improves Multilingual Neural Machine Translation

Multilingual neural machine translation models are trained to maximize t...

2 Yong Cheng, et al. ∙

research

∙ 02/23/2022

Using natural language prompts for machine translation

We explore the use of natural language prompts for controlling various a...

0 Xavier Garcia, et al. ∙

research

∙ 02/04/2022

Data Scaling Laws in NMT: The Effect of Noise and Architecture

In this work, we study the effect of varying the architecture and traini...

4 Yamini Bansal, et al. ∙

research

∙ 02/01/2022

Examining Scaling and Transfer of Language Model Architectures for Machine Translation

Natural language understanding and generation models follow one of the t...

7 Biao Zhang, et al. ∙

research

∙ 01/09/2022

Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning

Achieving universal translation between all human language pairs is the ...

8 Aditya Siddhant, et al. ∙

research

∙ 12/13/2021

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

Scaling language models with more data, compute and parameters has drive...

4 Nan Du, et al. ∙

research

∙ 10/08/2021

A Loss Curvature Perspective on Training Instability in Deep Learning

In this work, we study the evolution of the loss Hessian across many cla...

47 Justin Gilmer, et al. ∙

research

∙ 09/21/2021

Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents

Document-level neural machine translation (DocNMT) delivers coherent tra...

0 Biao Zhang, et al. ∙

research

∙ 09/19/2021

Towards Zero-Label Language Learning

This paper explores zero-label learning in Natural Language Processing (...

17 Zirui Wang, et al. ∙

research

∙ 09/16/2021

Scaling Laws for Neural Machine Translation

We present an empirical study of scaling properties of encoder-decoder T...

23 Behrooz Ghorbani, et al. ∙

research

∙ 09/13/2021

Evaluating Multiway Multilingual NMT in the Turkic Languages

Despite the increasing number of large and comprehensive machine transla...

11 Jamshidbek Mirzakhalov, et al. ∙

research

∙ 09/09/2021

A Large-Scale Study of Machine Translation in the Turkic Languages

Recent advances in neural machine translation (NMT) have pushed the qual...

0 Jamshidbek Mirzakhalov, et al. ∙

research

∙ 07/30/2021

Towards Universality in Multilingual Text Rewriting

In this work, we take the first steps towards building a universal rewri...

0 Xavier Garcia, et al. ∙

research

∙ 04/15/2021

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation

Machine learning has brought striking advances in multilingual natural l...

0 Sebastian Ruder, et al. ∙

research

∙ 03/22/2021

Quality at a Glance: An Audit of Web-Crawled Multilingual Datasets

With the success of large-scale pre-training and multilingual modeling i...

33 Isaac Caswell, et al. ∙

research

∙ 03/11/2021

Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution

We propose a straightforward vocabulary adaptation scheme to extend the ...

0 Xavier Garcia, et al. ∙

research

∙ 02/26/2021

Gradient-guided Loss Masking for Neural Machine Translation

To mitigate the negative effect of low quality training data on the perf...

0 Xinyi Wang, et al. ∙

research

∙ 10/23/2020

Rapid Domain Adaptation for Machine Translation with Monolingual Data

One challenge of machine translation is how to quickly adapt to unseen d...

5 Mahdis Mahdieh, et al. ∙

research

∙ 10/20/2020

Towards End-to-End In-Image Neural Machine Translation

In this paper, we offer a preliminary investigation into the task of in-...

0 Elman Mansimov, et al. ∙

research

∙ 10/20/2020

Complete Multilingual Neural Machine Translation

Multilingual Neural Machine Translation (MNMT) models are commonly train...

0 Markus Freitag, et al. ∙

research

∙ 10/15/2020

Explicit Alignment Objectives for Multilingual Bidirectional Encoders

Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) a...

0 Junjie Hu, et al. ∙

research

∙ 10/12/2020

Gradient Vaccine: Investigating and Improving Multi-task Optimization in Massively Multilingual Models

Massively multilingual models subsuming tens or even hundreds of languag...

3 Zirui Wang, et al. ∙

research

∙ 09/23/2020

Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages

Unsupervised translation has reached impressive performance on resource-...

0 Xavier Garcia, et al. ∙

research

∙ 06/30/2020

GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

Neural network scaling has been critical for improving the model quality...

7 Dmitry Lepikhin, et al. ∙

research

∙ 05/11/2020

Leveraging Monolingual Data with Self-Supervision for Multilingual Neural Machine Translation

Over the last few years two promising research directions in low-resourc...

0 Aditya Siddhant, et al. ∙

research

∙ 03/24/2020

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

Much recent progress in applications of machine learning models to NLP h...

0 Junjie Hu, et al. ∙

research

∙ 02/17/2020

On the Discrepancy between Density Estimation and Sequence Generation

Many sequence-to-sequence generation tasks, including machine translatio...

8 Jason Lee, et al. ∙

research

∙ 02/17/2020

Controlling Computation versus Quality for Neural Sequence Models

Most neural networks utilize the same amount of compute for every exampl...

0 Ankur Bapna, et al. ∙

research

∙ 10/30/2019

Fill in the Blanks: Imputing Missing Sentences for Larger-Context Neural Machine Translation

Most neural machine translation systems still translate sentences in iso...

0 Sébastien Jean, et al. ∙

research

∙ 10/15/2019

On the Importance of Word Boundaries in Character-level Neural Machine Translation

Neural Machine Translation (NMT) models generally perform translation us...

0 Duygu Ataman, et al. ∙

research

∙ 09/18/2019

Simple, Scalable Adaptation for Neural Machine Translation

Fine-tuning pre-trained Neural Machine Translation (NMT) models is the d...

0 Ankur Bapna, et al. ∙

research

∙ 09/13/2019

Adaptive Scheduling for Multi-Task Learning

To train neural machine translation models simultaneously on multiple ta...

29 Sébastien Jean, et al. ∙

research

∙ 09/05/2019

Investigating Multilingual NMT Representations at Scale

Multilingual Neural Machine Translation (NMT) models have yielded large ...

1 Sneha Reddy Kudugunta, et al. ∙

research

∙ 09/01/2019

Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

The recently proposed massively multilingual neural machine translation ...

0 Aditya Siddhant, et al. ∙

research

∙ 07/11/2019

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges

We introduce our efforts towards building a universal neural machine tra...

0 Naveen Arivazhagan, et al. ∙

research

∙ 06/27/2019

Findings of the First Shared Task on Machine Translation Robustness

We share the findings of the first shared task on improving robustness o...

0 Xian Li, et al. ∙

research

∙ 03/17/2019

The Missing Ingredient in Zero-Shot Neural Machine Translation

Multilingual Neural Machine Translation (NMT) models are capable of tran...

0 Naveen Arivazhagan, et al. ∙

research

∙ 02/28/2019

Massively Multilingual Neural Machine Translation

Multilingual neural machine translation (NMT) enables training a single ...

0 Roee Aharoni, et al. ∙

Orhan Firat

Featured Co-authors