Yuxiong He

is this you? claim profile

0 followers
Microsoft

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

Scalable training of large models (like BERT and GPT-3) requires careful...
02/04/2021 ∙ by Hanlin Tang, et al. ∙ 0 ∙ share

read it
ZeRO-Offload: Democratizing Billion-Scale Model Training

Large-scale model training has been a playing ground for a limited few r...
01/18/2021 ∙ by Jie Ren, et al. ∙ 0 ∙ share

read it
Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping

Recently, Transformer-based language models have demonstrated remarkable...
10/26/2020 ∙ by Minjia Zhang, et al. ∙ 0 ∙ share

read it
APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm

Adam is the important optimization algorithm to guarantee efficiency and...
08/26/2020 ∙ by Hanlin Tang, et al. ∙ 11 ∙ share

read it
LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory

The effectiveness of LSTM neural networks for popular tasks such as Auto...
11/04/2019 ∙ by Reza Yazdani, et al. ∙ 76 ∙ share

read it
ZeRO: Memory Optimization Towards Training A Trillion Parameter Models

Training large DL models with billions and potentially trillions of para...
10/04/2019 ∙ by Samyam Rajbhandari, et al. ∙ 1 ∙ share

read it
AntMan: Sparse Low-Rank Compression to Accelerate RNN inference

Wide adoption of complex RNN based models is hindered by their inference...
10/02/2019 ∙ by Samyam Rajbhandari, et al. ∙ 0 ∙ share

read it
Zoom: SSD-based Vector Search for Optimizing Accuracy, Latency and Memory

With the advancement of machine learning and deep learning, vector searc...
09/11/2018 ∙ by Minjia Zhang, et al. ∙ 0 ∙ share

read it
Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models

Neural language models (NLMs) have recently gained a renewed interest by...
06/11/2018 ∙ by Minjia Zhang, et al. ∙ 0 ∙ share

read it
Learning Intrinsic Sparse Structures within Long Short-Term Memory

Model compression is significant for the wide adoption of Recurrent Neur...
09/15/2017 ∙ by Wei Wen, et al. ∙ 0 ∙ share

read it

Yuxiong He

Featured Co-authors

Jianfeng Gao

Ji Liu

Ce Zhang

Yiran Chen

Hai Li

Fang Liu

Dong Li

Xiaodong Liu

Bin Hu

Hanlin Tang

Wei Wen

1-bit Adam: Communication Efficient Large-Scale Training with Adam's Convergence Speed

ZeRO-Offload: Democratizing Billion-Scale Model Training

Accelerating Training of Transformer-Based Language Models with Progressive Layer Dropping

APMSqueeze: A Communication Efficient Adam-Preconditioned Momentum SGD Algorithm

LSTM-Sharp: An Adaptable, Energy-Efficient Hardware Accelerator for Long Short-Term Memory

ZeRO: Memory Optimization Towards Training A Trillion Parameter Models

AntMan: Sparse Low-Rank Compression to Accelerate RNN inference

Zoom: SSD-based Vector Search for Optimizing Accuracy, Latency and Memory

Navigating with Graph Representations for Fast and Scalable Decoding of Neural Language Models

Learning Intrinsic Sparse Structures within Long Short-Term Memory