research
∙
06/19/2023
Co-design Hardware and Algorithm for Vector Search
Vector search has emerged as the foundation for large-scale information ...
share
research
∙
05/08/2023
ASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Gradient preconditioning is a key technique to integrate the second-orde...
research
∙
04/12/2023
An End-to-End Network for Upright Adjustment of Panoramic Images
Nowadays, panoramic images can be easily obtained by panoramic cameras. ...
research
∙
01/17/2023
AutoDDL: Automatic Distributed Deep Learning with Asymptotically Optimal Communication
Recent advances in deep learning base on growing model sizes and the nec...
research
∙
11/25/2022
PipeFisher: Efficient Training of Large Language Models Using Pipelining and Fisher Information Matrices
Pipeline parallelism enables efficient training of Large Language Models...
research
∙
09/14/2022
Efficient Quantized Sparse Matrix Operations on Tensor Cores
The exponentially growing model size drives the continued success of dee...
research
∙
09/03/2022
HammingMesh: A Network Topology for Large-Scale Deep Learning
Numerous microarchitectural optimizations unlocked tremendous processing...
research
∙
01/19/2022
Near-Optimal Sparse Allreduce for Distributed Deep Learning
Communication overhead is one of the major obstacles to train large deep...
research
∙
10/20/2021
A Data-Centric Optimization Framework for Machine Learning
Rapid progress in deep learning is leading to a diverse set of quickly c...
research
∙
07/14/2021
Chimera: Efficiently Training Large-Scale Neural Networks with Bidirectional Pipelines
Training large deep learning models at scale is very challenging. This p...
research
∙
06/29/2021
Flare: Flexible In-Network Allreduce
The allreduce operation is one of the most commonly used communication r...
research
∙
06/30/2020
Data Movement Is All You Need: A Case Study on Optimizing Transformers
Transformers have become widely used for language modeling and sequence ...
research
∙
05/18/2020
Deep Learning for Post-Processing Ensemble Weather Forecasts
Quantifying uncertainty in weather forecasts typically employs ensemble ...
research
∙
04/30/2020
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging
Deep learning at scale is dominated by communication time. Distributing ...
research
∙
11/02/2019
Predicting Weather Uncertainty with Deep Convnets
Modern weather forecast models perform uncertainty quantification using ...
research
∙
08/12/2019