research
∙
09/02/2023
RenAIssance: A Survey into AI Text-to-Image Generation in the Era of Large Model
Text-to-image generation (TTI) refers to the usage of models that could ...
share
research
∙
03/15/2023
A Comprehensive Study on Post-Training Quantization for Large Language Models
Post-training quantization () had been recently shown as a compromising ...
research
∙
03/13/2023
Scaling Vision-Language Models with Sparse Mixture of Experts
The field of natural language processing (NLP) has made significant stri...
research
∙
06/30/2022
DeepSpeed Inference: Enabling Efficient Inference of Transformer Models at Unprecedented Scale
The past several years have witnessed the success of transformer-based m...
research
∙
01/28/2022
Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model
Pretrained general-purpose language models can achieve state-of-the-art ...
research
∙
01/14/2022
DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale
As the training of giant dense models hits the boundary on the availabil...
research
∙
08/13/2021
Curriculum Learning: A Regularization Method for Efficient and Stable Billion-Scale GPT Model Pre-Training
Recent works have demonstrated great success in training high-capacity a...
research
∙
11/04/2019
LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory
The effectiveness of LSTM neural networks for popular tasks such as Auto...
research
∙
10/04/2019
ZeRO: Memory Optimization Towards Training A Trillion Parameter Models
Training large DL models with billions and potentially trillions of para...
research
∙
09/15/2017
