research
∙
05/23/2023
Coarse-to-Fine Contrastive Learning in Image-Text-Graph Space for Improved Vision-Language Compositionality
Contrastively trained vision-language models have achieved remarkable pr...
share
research
∙
11/11/2022
Speech-to-Speech Translation For A Real-world Unwritten Language
We study speech-to-speech translation (S2ST) that translates speech from...
research
∙
11/08/2022
SpeechMatrix: A Large-Scale Mined Corpus of Multilingual Speech-to-Speech Translations
We present SpeechMatrix, a large-scale multilingual corpus of speech-to-...
research
∙
05/30/2022
Prompting ELECTRA: Few-Shot Learning with Discriminative Pre-Trained Models
Pre-trained masked language models successfully perform few-shot learnin...
research
∙
05/24/2022
On the Role of Bidirectionality in Language Model Pre-Training
Prior work on language model pre-training has explored different archite...
research
∙
05/03/2022
Improving In-Context Few-Shot Learning via Self-Supervised Training
Self-supervised pretraining has made few-shot learning possible for many...
research
∙
12/20/2021
Efficient Large Scale Language Modeling with Mixtures of Experts
Mixture of Experts layers (MoEs) enable efficient scaling of language mo...
research
∙
12/20/2021
Few-shot Learning with Multilingual Language Models
Large-scale autoregressive language models such as GPT-3 are few-shot le...
research
∙
05/02/2021
Larger-Scale Transformers for Multilingual Masked Language Modeling
Recent work has demonstrated the effectiveness of cross-lingual language...
research
∙
11/03/2020
Supervised Contrastive Learning for Pre-trained Language Model Fine-tuning
State-of-the-art natural language understanding classification models fo...
research
∙
10/05/2020
Self-training Improves Pre-training for Natural Language Understanding
Unsupervised pre-training has led to much recent progress in natural lan...
research
∙
09/27/2020
Answering Complex Open-Domain Questions with Multi-Hop Dense Retrieval
We propose a simple and efficient multi-hop dense retrieval approach for...
research
∙
04/29/2020
General Purpose Text Embeddings from Pre-trained Language Models for Scalable Inference
The state of the art on many NLP tasks is currently achieved by large pr...
research
∙
12/20/2019
Pretrained Encyclopedia: Weakly Supervised Knowledge-Pretrained Language Model
Recent breakthroughs of pretrained language models have shown the effect...
research
∙
07/26/2019
RoBERTa: A Robustly Optimized BERT Pretraining Approach
Language model pretraining has led to significant performance gains but ...
research
∙
04/09/2019