research
∙
07/14/2023
MGit: A Model Versioning and Management System
Models derived from other models are extremely common in machine learnin...
share
research
∙
05/03/2023
Cheaply Evaluating Inference Efficiency Metrics for Autoregressive Transformer APIs
Large language models (LLMs) power many state-of-the-art systems in natu...
research
∙
11/29/2022
MegaBlocks: Efficient Sparse Training with Mixture-of-Experts
We present MegaBlocks, a system for efficient Mixture-of-Experts (MoE) t...
research
∙
11/16/2022
Holistic Evaluation of Language Models
Language models (LMs) are becoming the foundation for almost all major l...
research
∙
10/22/2021
Solving Large-Scale Granular Resource Allocation Problems Efficiently with POP
Resource allocation problems in many computer systems can be formulated ...
research
∙
04/13/2021
Don't Give Up on Large Optimization Problems; POP Them!
Resource allocation problems in many computer systems can be formulated ...
research
∙
04/09/2021
Efficient Large-Scale Language Model Training on GPU Clusters
Large language models have led to state-of-the-art accuracies across a r...
research
∙
04/01/2021
Allocation of Fungible Resources via a Fast, Scalable Price Discovery Method
We consider the problem of assigning or allocating resources to a set of...
research
∙
08/20/2020
Heterogeneity-Aware Cluster Scheduling Policies for Deep Learning Workloads
Specialized accelerators such as GPUs, TPUs, FPGAs, and custom ASICs hav...
research
∙
06/16/2020
Memory-Efficient Pipeline-Parallel DNN Training
Many state-of-the-art results in domains such as NLP and computer vision...
research
∙
10/02/2019
MLPerf Training Benchmark
Machine learning is experiencing an explosion of software and hardware s...
research
∙
06/03/2019
Willump: A Statistically-Aware End-to-end Optimizer for Machine Learning Inference
Machine learning (ML) has become increasingly important and performance-...
research
∙
06/08/2018
PipeDream: Fast and Efficient Pipeline Parallel DNN Training
PipeDream is a Deep Neural Network(DNN) training system for GPUs that pa...
research
∙
06/04/2018
Analysis of DAWNBench, a Time-to-Accuracy Machine Learning Performance Benchmark
The deep learning community has proposed optimizations spanning hardware...
research
∙
09/14/2017