research
∙
08/02/2023
DeepSpeed-Chat: Easy, Fast and Affordable RLHF Training of ChatGPT-like Models at All Scales
ChatGPT-like models have revolutionized various applications in artifici...
share
research
∙
03/15/2023
MCR-DL: Mix-and-Match Communication Runtime for Deep Learning
In recent years, the training requirements of many state-of-the-art Deep...
research
∙
06/30/2022
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
The past several years have witnessed the success of transformer-based m...
research
∙
01/14/2022
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
As the training of giant dense models hits the boundary on the availabil...
research
∙
04/16/2021
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
In the last three years, the largest dense deep learning models have gro...
research
∙
10/04/2019