research
∙
05/10/2022
Reducing Activation Recomputation in Large Transformer Models
Training large transformer models is one of the most important computati...
share
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
09/17/2019
Megatron-LM: Training Multi-Billion Parameter Language Models Using GPU Model Parallelism
Recent work in unsupervised language modeling demonstrates that training...
research
∙
12/08/2015
Deep Speech 2: End-to-End Speech Recognition in English and Mandarin
We show that an end-to-end deep learning approach can be used to recogni...
research
∙
12/17/2014