research
∙
09/12/2023
A Distributed Data-Parallel PyTorch Implementation of the Distributed Shampoo Optimizer for Training Neural Networks At-Scale
Shampoo is an online and stochastic optimization algorithm belonging to ...
share
research
∙
04/19/2023
MTrainS: Improving DLRM training efficiency using heterogeneous memories
Recommendation models are very large, requiring terabytes (TB) of memory...
research
∙
03/28/2022
Learning to Collide: Recommendation System Model Compression with Learned Hash Functions
A key characteristic of deep recommendation models is the immense memory...
research
∙
02/01/2022
TopoOpt: Optimizing the Network Topology for Distributed DNN Training
We explore a novel approach for building DNN training clusters using com...
research
∙
10/25/2021
Differentiable NAS Framework and Application to Ads CTR Prediction
Neural architecture search (NAS) methods aim to automatically find the o...
research
∙
10/21/2021
Supporting Massive DLRM Inference Through Software Defined Memory
Deep Learning Recommendation Models (DLRM) are widespread, account for a...
research
∙
04/12/2021
Software-Hardware Co-design for Fast and Scalable Training of Deep Learning Recommendation Models
Deep learning recommendation models (DLRMs) are used across many busines...
research
∙
10/17/2020
Check-N-Run: A Checkpointing System for Training Recommendation Models
Checkpoints play an important role in training recommendation systems at...
research
∙
03/20/2020
Deep Learning Training in Facebook Data Centers: Design of Scale-up and Scale-out Systems
Large-scale training is important to ensure high performance and accurac...
research
∙
01/15/2020
SEERL: Sample Efficient Ensemble Reinforcement Learning
Ensemble learning is a very prevalent method employed in machine learnin...
research
∙
12/30/2019
RecNMP: Accelerating Personalized Recommendation with Near-Memory Processing
Personalized recommendation systems leverage deep learning models and ac...
research
∙
09/25/2019
Mixed Dimension Embeddings with Application to Memory-Efficient Recommendation Systems
In many real-world applications, e.g. recommendation systems, certain it...
research
∙
09/04/2019
Compositional Embeddings Using Complementary Partitions for Memory-Efficient Recommendation Systems
Modern deep learning-based recommendation systems exploit hundreds to th...
research
∙
06/06/2019
The Architectural Implications of Facebook's DNN-based Personalized Recommendation
The widespread application of deep learning has changed the landscape of...
research
∙
05/31/2019
Deep Learning Recommendation Model for Personalization and Recommendation Systems
With the advent of deep learning, neural network-based recommendation mo...
research
∙
05/29/2019
A Study of BFLOAT16 for Deep Learning Training
This paper presents the first comprehensive empirical study demonstratin...
research
∙
10/26/2018
Efficient Distributed Hessian Free Algorithm for Large-scale Empirical Risk Minimization via Accumulating Sample Strategy
In this paper, we propose a Distributed Accumulated Newton Conjugate gra...
research
∙
02/15/2018
A Progressive Batching L-BFGS Method for Machine Learning
The standard L-BFGS method relies on gradient approximations that are no...
research
∙
02/03/2018
Mixed Precision Training of Convolutional Neural Networks using Integer Operations
The state-of-the-art (SOTA) for mixed precision training is dominated by...
research
∙
01/24/2018
On Scale-out Deep Learning Training for Cloud and HPC
The exponential growth in use of large deep neural networks has accelera...
research
∙
07/20/2017
RAIL: Risk-Averse Imitation Learning
Imitation learning algorithms learn viable policies by imitating an expe...
research
∙
07/15/2017
Ternary Residual Networks
Sub-8-bit representation of DNNs incur some discernible loss of accuracy...
research
∙
05/02/2017
Ternary Neural Networks with Fine-Grained Quantization
We propose a novel fine-grained quantization (FGQ) method to ternarize p...
research
∙
01/31/2017
Mixed Low-precision Deep Learning Inference using Dynamic Fixed Point
We propose a cluster-based quantization method to convert pre-trained fu...
research
∙
11/16/2010