
A Scalable Concurrent Algorithm for Dynamic Connectivity
Dynamic Connectivity is a fundamental algorithmic graph problem, motivat...
read it

NUQSGD: Provably Communicationefficient Dataparallel SGD via Nonuniform Quantization
As the size and complexity of models and datasets grow, so does the need...
read it

Waitfree approximate agreement on graphs
Approximate agreement is one of the few variants of consensus that can b...
read it

Fast Graphical Population Protocols
Let G be a graph on n nodes. In the stochastic population protocol model...
read it

CommunicationEfficient Distributed Optimization with Quantized Preconditioners
We investigate fast and communicationefficient algorithms for the class...
read it

Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks
The growing energy and performance costs of deep learning have driven th...
read it

ByzantineResilient NonConvex Stochastic Gradient Descent
We study adversaryresilient stochastic distributed optimization, in whi...
read it

Adaptive Gradient Quantization for DataParallel SGD
Many communicationefficient variants of SGD use gradient quantization s...
read it

Improved Communication Lower Bounds for Distributed Optimisation
Motivated by the interest in communicationefficient methods for distrib...
read it

The SplayList: A DistributionAdaptive Concurrent SkipList
The design and implementation of efficient concurrent data structures ha...
read it

Fast General Distributed Transactions with Opacity using Global Time
Transactions can simplify distributed applications by hiding data distri...
read it

Stochastic Gradient Langevin with Delayed Gradients
Stochastic Gradient Langevin Dynamics (SGLD) ensures strong guarantees w...
read it

Breaking (Global) Barriers in Parallel Stochastic Optimization with WaitAvoiding Group Averaging
Deep learning at scale is dominated by communication time. Distributing ...
read it

WoodFisher: Efficient secondorder approximations for model compression
Secondorder information, in the form of Hessian or InverseHessianvec...
read it

Efficiency Guarantees for Parallel Incremental Algorithms under Relaxed Schedulers
Several classic problems in graph processing and computational geometry ...
read it

Dynamic Averaging Load Balancing on Cycles
We consider the following dynamic loadbalancing process: given an under...
read it

Robust Comparison in Population Protocols
There has recently been a surge of interest in the computational and com...
read it

Relaxed Scheduling for Scalable Belief Propagation
The ability to leverage largescale hardware parallelism has been one of...
read it

On the Sample Complexity of Adversarial MultiSource PAC Learning
We study the problem of learning from multiple untrusted data sources, a...
read it

Distributed Mean Estimation with Optimal Error Bounds
Motivated by applications to distributed optimization and machine learni...
read it

Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent
Machine learning has made tremendous progress in recent years, with mode...
read it

Analysis and Evaluation of NonBlocking Interpolation Search Trees
We start by summarizing the recently proposed implementation of the firs...
read it

In Search of the Fastest Concurrent UnionFind Algorithm
UnionFind (or DisjointSet Union) is one of the fundamental problems in...
read it

PopSGD: Decentralized Stochastic Gradient Descent in the Population Model
The population model is a standard way to represent largescale decentra...
read it

Powerset Convolutional Neural Networks
We present a novel class of convolutional neural networks (CNNs) for set...
read it

Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations
Load imbalance pervasively exists in distributed deep learning training ...
read it

SysML: The New Frontier of Machine Learning Systems
Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
read it

Why ExtensionBased Proofs Fail
We prove that a class of fundamental shared memory tasks are not amenabl...
read it

Distributed Learning over Unreliable Networks
Most of today's distributed machine learning systems assume reliable ne...
read it

The Convergence of Sparsified Gradient Methods
Distributed training of massive machine learning models, in particular d...
read it

Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms
There has been significant progress in understanding the parallelism inh...
read it

Distributionally Linearizable Data Structures
Relaxed concurrent data structures have become increasingly popular, due...
read it

The Transactional Conflict Problem
The transactional conflict problem arises in transactional systems whene...
read it

Byzantine Stochastic Gradient Descent
This paper studies the problem of distributed stochastic optimization in...
read it

The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory
Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine ...
read it

SparCML: HighPerformance Sparse Communication for Machine Learning
One of the main drivers behind the rapid recent advances in machine lear...
read it

Model compression via distillation and quantization
Deep neural networks (DNNs) continue to make significant advances, solvi...
read it

Compressive Sensing with Low Precision Data Representation: Radio Astronomy and Beyond
Modern scientific instruments produce vast amounts of data, which can ov...
read it

DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation
It is safe to assume that, for the foreseeable future, machine learning,...
read it

The ZipML Framework for Training Models with EndtoEnd Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning
Recently there has been significant interest in training machinelearnin...
read it
Dan Alistarh
is this you? claim profile