Melvin Johnson

research

∙ 05/19/2023

XTREME-UP: A User-Centric Scarce-Data Benchmark for Under-Represented Languages

Data scarcity is a crucial issue for the development of highly multiling...

0 Sebastian Ruder, et al. ∙

research

∙ 02/02/2023

The unreasonable effectiveness of few-shot learning for machine translation

We demonstrate the potential of few-shot translation systems, trained wi...

0 Xavier Garcia, et al. ∙

research

∙ 12/19/2022

Mu^2SLAM: Multitask, Multilingual Speech and Language Models

We present Mu^2SLAM, a multilingual sequence-to-sequence model pre-train...

0 Yong Cheng, et al. ∙

research

∙ 03/21/2022

XTREME-S: Evaluating Cross-lingual Speech Representations

We introduce XTREME-S, a new benchmark to evaluate universal cross-lingu...

0 Alexis Conneau, et al. ∙

research

∙ 02/03/2022

mSLAM: Massively multilingual joint pre-training for speech and text

We present mSLAM, a multilingual Speech and LAnguage Model that learns c...

0 Ankur Bapna, et al. ∙

research

∙ 12/16/2021

DOCmT5: Document-Level Pretraining of Multilingual Language Models

In this paper, we introduce DOCmT5, a multilingual sequence-to-sequence ...

0 Chia-Hsuan Lee, et al. ∙

research

∙ 10/20/2021

SLAM: A Unified Encoder for Speech and Language Modeling via Speech-Text Joint Pre-Training

Unsupervised pre-training is now the predominant approach for both text ...

26 Ankur Bapna, et al. ∙

research

∙ 09/21/2021

Multilingual Document-Level Translation Enables Zero-Shot Transfer From Sentences to Documents

Document-level neural machine translation (DocNMT) delivers coherent tra...

0 Biao Zhang, et al. ∙

research

∙ 09/09/2021

HintedBT: Augmenting Back-Translation with Quality and Transliteration Hints

Back-translation (BT) of target monolingual corpora is a widely used dat...

0 Sahana Ramnath, et al. ∙

research

∙ 06/05/2021

MergeDistill: Merging Pre-trained Language Models using Distillation

Pre-trained multilingual language models (LMs) have achieved state-of-th...

0 Simran Khanuja, et al. ∙

research

∙ 06/03/2021

nmT5 – Is parallel data still relevant for pre-training massively multilingual language models?

Recently, mT5 - a massively multilingual version of T5 - leveraged a uni...

0 Mihir Kale, et al. ∙

research

∙ 04/15/2021

XTREME-R: Towards More Challenging and Nuanced Multilingual Evaluation

Machine learning has brought striking advances in multilingual natural l...

0 Sebastian Ruder, et al. ∙

research

∙ 02/26/2021

Gradient-guided Loss Masking for Neural Machine Translation

To mitigate the negative effect of low quality training data on the perf...

0 Xinyi Wang, et al. ∙

research

∙ 02/12/2021

They, Them, Theirs: Rewriting with Gender-Neutral English

Responsible development of technology involves applications being inclus...

0 Tony Sun, et al. ∙

research

∙ 01/21/2021

Distilling Large Language Models into Tiny and Effective Students using pQRNN

Large pre-trained multilingual models like mBERT, XLM-R achieve state of...

0 Prabhu Kaliamoorthi, et al. ∙

research

∙ 10/24/2020

Rethinking embedding coupling in pre-trained language models

We re-evaluate the standard practice of sharing weights between input an...

0 Hyung Won Chung, et al. ∙

research

∙ 10/15/2020

Explicit Alignment Objectives for Multilingual Bidirectional Encoders

Pre-trained cross-lingual encoders such as mBERT (Devlin et al., 2019) a...

0 Junjie Hu, et al. ∙

research

∙ 03/24/2020

XTREME: A Massively Multilingual Multi-task Benchmark for Evaluating Cross-lingual Generalization

Much recent progress in applications of machine learning models to NLP h...

0 Junjie Hu, et al. ∙

research

∙ 09/13/2019

Adaptive Scheduling for Multi-Task Learning

To train neural machine translation models simultaneously on multiple ta...

29 Sébastien Jean, et al. ∙

research

∙ 09/01/2019

Evaluating the Cross-Lingual Effectiveness of Massively Multilingual Neural Machine Translation

The recently proposed massively multilingual neural machine translation ...

0 Aditya Siddhant, et al. ∙

research

∙ 08/31/2019

Small and Practical BERT Models for Sequence Labeling

We propose a practical scheme to train a single multilingual sequence la...

0 Henry Tsai, et al. ∙

research

∙ 07/11/2019

Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges

We introduce our efforts towards building a universal neural machine tra...

0 Naveen Arivazhagan, et al. ∙

research

∙ 04/12/2019

Direct speech-to-speech translation with a sequence-to-sequence model

We present an attention-based sequence-to-sequence neural network which ...

0 Ye Jia, et al. ∙

research

∙ 03/17/2019

The Missing Ingredient in Zero-Shot Neural Machine Translation

Multilingual Neural Machine Translation (NMT) models are capable of tran...

0 Naveen Arivazhagan, et al. ∙

research

∙ 02/28/2019

Massively Multilingual Neural Machine Translation

Multilingual neural machine translation (NMT) enables training a single ...

0 Roee Aharoni, et al. ∙

research

∙ 02/21/2019

Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling

Lingvo is a Tensorflow framework offering a complete solution for collab...

13 Jonathan Shen, et al. ∙

research

∙ 11/05/2018

Leveraging Weakly Supervised Data to Improve End-to-End Speech-to-Text Translation

End-to-end Speech Translation (ST) models have many potential advantages...

0 Ye Jia, et al. ∙

research

∙ 09/12/2018

Zero-Shot Cross-lingual Classification Using Multilingual Neural Machine Translation

Transferring representations from large supervised tasks to downstream t...

0 Akiko Eriguchi, et al. ∙

research

∙ 04/26/2018

The Best of Both Worlds: Combining Recent Advances in Neural Machine Translation

The past year has witnessed rapid advances in sequence-to-sequence (seq2...

0 Mia Xu Chen, et al. ∙

research

∙ 11/14/2016

Google's Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation

We propose a simple solution to use a single Neural Machine Translation ...

0 Melvin Johnson, et al. ∙

research

∙ 09/26/2016

Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation

Neural Machine Translation (NMT) is an end-to-end learning approach for ...

0 Yonghui Wu, et al. ∙

Melvin Johnson

Featured Co-authors