research
∙
05/29/2023
Brainformers: Trading Simplicity for Efficiency
Transformers are central to recent successes in natural language process...
share
research
∙
11/09/2022
Efficiently Scaling Transformer Inference
We study the problem of efficient generative inference for Transformer m...
research
∙
09/04/2022
A Review of Sparse Expert Models in Deep Learning
Sparse expert models are a thirty-year old concept re-emerging as a popu...
research
∙
05/25/2022
An Evolutionary Approach to Dynamic Introduction of Tasks in Large-scale Multitask Learning Systems
Multitask learning assumes that models capable of learning from multiple...
research
∙
05/22/2022
muNet: Evolving Pretrained Deep Neural Networks into Scalable Auto-tuning Multitask Systems
Most uses of machine learning today involve training a model from scratc...
research
∙
04/11/2022
The Carbon Footprint of Machine Learning Training Will Plateau, Then Shrink
Machine Learning (ML) workloads have rapidly grown in importance, but ra...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
03/23/2022
Pathways: Asynchronous Distributed Dataflow for ML
We present the design of a new large scale orchestration layer for accel...
research
∙
02/17/2022
Designing Effective Sparse Expert Models
Scale has opened new frontiers in natural language processing – but at a...
research
∙
04/21/2021
Carbon Emissions and Large Neural Network Training
The computation demand for machine learning (ML) has grown rapidly recen...
research
∙
10/08/2020
Interlocking Backpropagation: Improving depthwise model-parallelism
The number of parameters in state of the art neural networks has drastic...
research
∙
04/22/2020
Chip Placement with Deep Reinforcement Learning
In this work, we present a learning-based approach to chip placement, on...
research
∙
03/29/2019
SysML: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
research
∙
05/04/2018
Dynamic Control Flow in Large-Scale Machine Learning
Many recent machine learning models rely on fine-grained dynamic control...
research
∙
02/09/2018
Efficient Neural Architecture Search via Parameter Sharing
We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...
research
∙
02/09/2018
Efficient Neural Architecture Search via Parameters Sharing
We propose Efficient Neural Architecture Search (ENAS), a fast and inexp...
research
∙
01/24/2018
Scalable and accurate deep learning for electronic health records
Predictive modeling with electronic health record (EHR) data is anticipa...
research
∙
06/13/2017
Device Placement Optimization with Reinforcement Learning
The past few years have witnessed a growth in size and computational req...
research
∙
01/23/2017
Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer
The capacity of a neural network to absorb information is limited by its...
research
∙
03/09/2015
Distilling the Knowledge in a Neural Network
A very simple way to improve the performance of almost any machine learn...
research
∙
12/19/2013
Using Web Co-occurrence Statistics for Improving Image Categorization
Object recognition and localization are important tasks in computer visi...
research
∙
12/29/2011