James Demmel | DeepAI

DeepAI

Featured Co-authors

Cho-Jui Hsieh
138 publications
Torsten Hoefler
117 publications
Michael W. Mahoney
115 publications
Jing Li
105 publications
Kurt Keutzer
100 publications
Zhao Zhang
79 publications
Huan Zhang
67 publications
Yang You
49 publications
Aydin Buluc
37 publications
Mingxing Tan
32 publications
Quoc Le
29 publications

∙ 01/29/2023

Fast Exact Leverage Score Sampling from Khatri-Rao Products with Applications to Tensor Decomposition

We present a data structure to randomly sample rows from the Khatri-Rao ...

Vivek Bharadwaj, et al. ∙

∙ 10/11/2022

Distributed-Memory Randomized Algorithms for Sparse Tensor CP Decomposition

Low-rank Candecomp / PARAFAC (CP) Decomposition is a powerful tool for t...

Vivek Bharadwaj, et al. ∙

∙ 07/19/2022

Proposed Consistent Exception Handling for the BLAS and LAPACK

Numerical exceptions, which may be caused by overflow, operations like d...

James Demmel, et al. ∙

∙ 04/18/2022

Communication Bounds for Convolutional Neural Networks

Convolutional neural networks (CNNs) are important in a wide variety of ...

Anthony Chen, et al. ∙

∙ 03/15/2022

Distributed-Memory Sparse Kernels for Machine Learning

Sampled Dense Times Dense Matrix Multiplication (SDDMM) and Sparse Times...

Vivek Bharadwaj, et al. ∙

∙ 05/05/2021

CoSA: Scheduling by Constrained Optimization for Spatial Accelerators

Recent advances in Deep Neural Networks (DNNs) have led to active develo...

Qijing Huang, et al. ∙

∙ 11/16/2020

Avoiding Communication in Logistic Regression

Stochastic gradient descent (SGD) is one of the most widely used optimiz...

Aditya Devarakonda, et al. ∙

∙ 10/30/2020

Training EfficientNets at Supercomputer Scale: 83 Accuracy in One Hour

EfficientNets are a family of state-of-the-art image classification mode...

Arissa Wongpanich, et al. ∙

∙ 06/15/2020

The Limit of the Batch Size

Large-batch training is an efficient approach for current distributed de...

Yang You, et al. ∙

∙ 02/28/2020

Communication-Optimal Tilings for Projective Nested Loops with Arbitrary Bounds

Reducing communication - either between levels of a memory hierarchy or ...

Grace Dinh, et al. ∙

∙ 11/20/2019

Auto-Precision Scaling for Distributed Deep Learning

In recent years, large-batch optimization is becoming the key of distrib...

Ruobing Han, et al. ∙

∙ 10/01/2019

An improved analysis and unified perspective on deterministic and randomized low rank matrix approximations

We introduce a Generalized LU-Factorization (GLU) for low-rank matrix ap...

James Demmel, et al. ∙

∙ 09/14/2019

A Generalized Randomized Rank-Revealing Factorization

We introduce a Generalized Randomized QR-decomposition that may be appli...

Grey Ballard, et al. ∙

∙ 04/01/2019

Reducing BERT Pre-Training Time from 3 Days to 76 Minutes

Large-batch training is key to speeding up deep neural network training ...

Yang You, et al. ∙

∙ 01/24/2019

Large-Batch Training for LSTM and Beyond

Large-batch training approaches have enabled researchers to utilize larg...

Yang You, et al. ∙

∙ 05/14/2018

A 3D Parallel Algorithm for QR Decomposition

Interprocessor communication often dominates the runtime of large matrix...

Grey Ballard, et al. ∙

∙ 05/01/2018

Accurate, Fast and Scalable Kernel Ridge Regression on Parallel and Distributed Systems

We propose two new methods to address the weak scaling problems of KRR: ...

Yang You, et al. ∙

∙ 02/19/2018

Communication-Optimal Convolutional Neural Nets

Efficiently executing convolutional neural nets (CNNs) is important in m...

James Demmel, et al. ∙

∙ 12/17/2017

Avoiding Synchronization in First-Order Methods for Sparse Convex Optimization

Parallel computing has played an important role in speeding up convex op...

Aditya Devarakonda, et al. ∙

∙ 10/24/2017

Avoiding Communication in Proximal Methods for Convex Optimization Problems

The fast iterative soft thresholding algorithm (FISTA) is used to solve ...

Saeed Soori, et al. ∙

∙ 09/14/2017

ImageNet Training in Minutes

Finishing 90-epoch ImageNet-1k training with ResNet-50 on a NVIDIA M40 G...

Yang You, et al. ∙

∙ 07/14/2017

Communication Lower Bounds of Bilinear Algorithms for Symmetric Tensor Contractions

Accurate numerical calculations of electronic structure are often domina...

Edgar Solomonik, et al. ∙