research
∙
09/09/2023
MADLAD-400: A Multilingual And Document-Level Large Audited Dataset
We introduce MADLAD-400, a manually audited, general domain 3T token mon...
share
research
∙
05/19/2023
Cross-Lingual Supervision improves Large Language Models Pre-training
The recent rapid progress in pre-training Large Language Models has reli...
research
∙
04/18/2023
UniMax: Fairer and more Effective Language Sampling for Large-Scale Multilingual Pretraining
Pretrained multilingual large language models have typically used heuris...
research
∙
02/19/2023
Scaling Laws for Multilingual Neural Machine Translation
In this work, we provide a large-scale empirical study of the scaling pr...
research
∙
02/03/2023
Measuring The Impact Of Programming Language Distribution
Current benchmarks for evaluating neural code models focus on only a sma...
research
∙
02/02/2023
The unreasonable effectiveness of few-shot learning for machine translation
We demonstrate the potential of few-shot translation systems, trained wi...
research
∙
01/24/2023
Interactive-Chain-Prompting: Ambiguity Resolution for Crosslingual Conditional Generation with Interaction
Crosslingual conditional generation (e.g., machine translation) has long...
research
∙
10/20/2022
Transcending Scaling Laws with 0.1
Scaling language models improves performance but comes with significant ...
research
∙
10/01/2022
FRMT: A Benchmark for Few-Shot Region-Aware Machine Translation
We present FRMT, a new dataset and evaluation benchmark for Few-shot Reg...
research
∙
05/09/2022
Building Machine Translation Systems for the Next Thousand Languages
In this paper we share findings from our effort to build practical machi...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
03/31/2022
Scaling Up Models and Data with and
Recent neural network-based language models have benefited greatly from ...
research
∙
02/23/2022
Using natural language prompts for machine translation
We explore the use of natural language prompts for controlling various a...
research
∙
02/01/2022
Examining Scaling and Transfer of Language Model Architectures for Machine Translation
Natural language understanding and generation models follow one of the t...
research
∙
01/09/2022
Towards the Next 1000 Languages in Multilingual Machine Translation: Exploring the Synergy Between Supervised and Self-Supervised Learning
Achieving universal translation between all human language pairs is the ...
research
∙
10/14/2021
Few-shot Controllable Style Transfer for Low-Resource Settings: A Study in Indian Languages
Style transfer is the task of rewriting an input sentence into a target ...
research
∙
09/16/2021
Scaling Laws for Neural Machine Translation
We present an empirical study of scaling properties of encoder-decoder T...
research
∙
07/30/2021
Towards Universality in Multilingual Text Rewriting
In this work, we take the first steps towards building a universal rewri...
research
∙
03/11/2021
Towards Continual Learning for Multilingual Machine Translation via Vocabulary Substitution
We propose a straightforward vocabulary adaptation scheme to extend the ...
research
∙
09/23/2020
Harnessing Multilinguality in Unsupervised Machine Translation for Rare Languages
Unsupervised translation has reached impressive performance on resource-...
research
∙
08/25/2020
Machine learning applied in the multi-scale 3D stress modelling
This paper proposes a methodology to estimate stress in the subsurface b...
research
∙
02/07/2020