research
∙
12/15/2022
A Study on the Intersection of GPU Utilization and CNN Inference
There has been significant progress in developing neural network archite...
share
research
∙
02/02/2022
Harmony: Overcoming the hurdles of GPU memory capacity to train massive DNN models on commodity servers
Deep neural networks (DNNs) have grown exponentially in complexity and s...
research
∙
10/12/2021
Synergy: Resource Sensitive DNN Scheduling in Multi-Tenant Clusters
Training Deep Neural Networks (DNNs) is a widely popular workload in bot...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
08/20/2020
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads
Specialized accelerators such as GPUs, TPUs, FPGAs, and custom ASICs hav...
research
∙
07/14/2020
Analyzing and Mitigating Data Stalls in DNN Training
Training Deep Neural Networks (DNNs) is resource-intensive and time-cons...
research
∙
06/29/2020
Efficient Algorithms for Device Placement of DNN Graph Operators
Modern machine learning workloads use large models, with complex structu...
research
∙
06/16/2020
Memory-Efficient Pipeline-Parallel DNN Training
Many state-of-the-art results in domains such as NLP and computer vision...
research
∙
06/05/2020
Daydream: Accurately Estimating the Efficacy of Optimizations for DNN Training
Modern deep neural network (DNN) training jobs use complex and heterogen...
research
∙
10/11/2019
Blink: Fast and Generic Collectives for Distributed ML
Model parameter synchronization across GPUs introduces high overheads fo...
research
∙
10/01/2019
The Non-IID Data Quagmire of Decentralized Machine Learning
Many large-scale machine learning (ML) applications need to train ML mod...
research
∙
07/02/2019
Themis: Fair and Efficient GPU Cluster Scheduling for Machine Learning Workloads
Modern distributed machine learning (ML) training workloads benefit sign...
research
∙
01/17/2019
Analysis of Large-Scale Multi-Tenant GPU Clusters for DNN Training Workloads
With widespread advances in machine learning, a number of large enterpri...
research
∙
06/08/2018
PipeDream: Fast and Efficient Pipeline Parallel DNN Training
PipeDream is a Deep Neural Network(DNN) training system for GPUs that pa...
research
∙
05/21/2018
Parameter Hub: a Rack-Scale Parameter Server for Distributed Deep Neural Network Training
Distributed deep neural network (DDNN) training constitutes an increasin...
research
∙
03/16/2018
TBD: Benchmarking and Analyzing Deep Neural Network Training
The recent popularity of deep neural networks (DNNs) has generated a lot...
research
∙
01/30/2018