research
∙
08/15/2023
RAVEN: In-Context Learning with Retrieval Augmented Encoder-Decoder Language Models
In this paper, we investigate the in-context learning ability of retriev...
share
research
∙
09/12/2022
FP8 Formats for Deep Learning
FP8 is a natural progression for accelerating deep learning training inf...
research
∙
05/10/2022
Reducing Activation Recomputation in Large Transformer Models
Training large transformer models is one of the most important computati...
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
07/05/2021
Long-Short Transformer: Efficient Transformers for Language and Vision
Transformers have achieved success in both language and vision domains. ...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
03/02/2020
Style Example-Guided Text Generation using Generative Adversarial Transformers
We introduce a language generative model framework for generating a styl...
research
∙
02/22/2020
Training Question Answering Models From Synthetic Data
Question and answer generation is a data augmentation method that aims t...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using GPU Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
02/25/2017
