b'Dan Alistarh'

A Scalable Concurrent Algorithm for Dynamic Connectivity

Dynamic Connectivity is a fundamental algorithmic graph problem, motivat...
05/17/2021 ∙ by Alexander Fedorov, et al. ∙ 0 ∙ share

read it
NUQSGD: Provably Communication-efficient Data-parallel SGD via Nonuniform Quantization

As the size and complexity of models and datasets grow, so does the need...
04/28/2021 ∙ by Ali Ramezani-Kebrya, et al. ∙ 0 ∙ share

read it
Wait-free approximate agreement on graphs

Approximate agreement is one of the few variants of consensus that can b...
03/16/2021 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Fast Graphical Population Protocols

Let G be a graph on n nodes. In the stochastic population protocol model...
02/17/2021 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Communication-Efficient Distributed Optimization with Quantized Preconditioners

We investigate fast and communication-efficient algorithms for the class...
02/14/2021 ∙ by Foivos Alimisis, et al. ∙ 0 ∙ share

read it
Sparsity in Deep Learning: Pruning and growth for efficient inference and training in neural networks

The growing energy and performance costs of deep learning have driven th...
01/31/2021 ∙ by Torsten Hoefler, et al. ∙ 85 ∙ share

read it
Byzantine-Resilient Non-Convex Stochastic Gradient Descent

We study adversary-resilient stochastic distributed optimization, in whi...
12/28/2020 ∙ by Zeyuan Allen-Zhu, et al. ∙ 0 ∙ share

read it
Adaptive Gradient Quantization for Data-Parallel SGD

Many communication-efficient variants of SGD use gradient quantization s...
10/23/2020 ∙ by Fartash Faghri, et al. ∙ 0 ∙ share

read it
Improved Communication Lower Bounds for Distributed Optimisation

Motivated by the interest in communication-efficient methods for distrib...
10/16/2020 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
The Splay-List: A Distribution-Adaptive Concurrent Skip-List

The design and implementation of efficient concurrent data structures ha...
08/03/2020 ∙ by Vitaly Aksenov, et al. ∙ 0 ∙ share

read it
Fast General Distributed Transactions with Opacity using Global Time

Transactions can simplify distributed applications by hiding data distri...
06/25/2020 ∙ by Alex Shamis, et al. ∙ 0 ∙ share

read it
Stochastic Gradient Langevin with Delayed Gradients

Stochastic Gradient Langevin Dynamics (SGLD) ensures strong guarantees w...
06/12/2020 ∙ by Vyacheslav Kungurtsev, et al. ∙ 0 ∙ share

read it
Breaking (Global) Barriers in Parallel Stochastic Optimization with Wait-Avoiding Group Averaging

Deep learning at scale is dominated by communication time. Distributing ...
04/30/2020 ∙ by Shigang Li, et al. ∙ 0 ∙ share

read it
WoodFisher: Efficient second-order approximations for model compression

Second-order information, in the form of Hessian- or Inverse-Hessian-vec...
04/29/2020 ∙ by Sidak Pal Singh, et al. ∙ 0 ∙ share

read it
Efficiency Guarantees for Parallel Incremental Algorithms under Relaxed Schedulers

Several classic problems in graph processing and computational geometry ...
03/20/2020 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Dynamic Averaging Load Balancing on Cycles

We consider the following dynamic load-balancing process: given an under...
03/20/2020 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Robust Comparison in Population Protocols

There has recently been a surge of interest in the computational and com...
03/13/2020 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Relaxed Scheduling for Scalable Belief Propagation

The ability to leverage large-scale hardware parallelism has been one of...
02/25/2020 ∙ by Vitaly Aksenov, et al. ∙ 0 ∙ share

read it
On the Sample Complexity of Adversarial Multi-Source PAC Learning

We study the problem of learning from multiple untrusted data sources, a...
02/24/2020 ∙ by Nikola Konstantinov, et al. ∙ 0 ∙ share

read it
Distributed Mean Estimation with Optimal Error Bounds

Motivated by applications to distributed optimization and machine learni...
02/21/2020 ∙ by Dan Alistarh, et al. ∙ 11 ∙ share

read it
Elastic Consistency: A General Consistency Model for Distributed Stochastic Gradient Descent

Machine learning has made tremendous progress in recent years, with mode...
01/16/2020 ∙ by Dan Alistarh, et al. ∙ 7 ∙ share

read it
Analysis and Evaluation of Non-Blocking Interpolation Search Trees

We start by summarizing the recently proposed implementation of the firs...
01/02/2020 ∙ by Aleksandar Prokopec, et al. ∙ 0 ∙ share

read it
In Search of the Fastest Concurrent Union-Find Algorithm

Union-Find (or Disjoint-Set Union) is one of the fundamental problems in...
11/14/2019 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
PopSGD: Decentralized Stochastic Gradient Descent in the Population Model

The population model is a standard way to represent large-scale decentra...
10/27/2019 ∙ by Giorgi Nadiradze, et al. ∙ 0 ∙ share

read it
Powerset Convolutional Neural Networks

We present a novel class of convolutional neural networks (CNNs) for set...
09/05/2019 ∙ by Chris Wendler, et al. ∙ 0 ∙ share

read it
Taming Unbalanced Training Workloads in Deep Learning with Partial Collective Operations

Load imbalance pervasively exists in distributed deep learning training ...
08/12/2019 ∙ by Shigang Li, et al. ∙ 0 ∙ share

read it
SysML: The New Frontier of Machine Learning Systems

Machine learning (ML) techniques are enjoying rapidly increasing adoptio...
03/29/2019 ∙ by Alexander Ratner, et al. ∙ 0 ∙ share

read it
Why Extension-Based Proofs Fail

We prove that a class of fundamental shared memory tasks are not amenabl...
11/04/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Distributed Learning over Unreliable Networks

Most of today's distributed machine learning systems assume reliable ne...
10/17/2018 ∙ by Hanlin Tang, et al. ∙ 0 ∙ share

read it
The Convergence of Sparsified Gradient Methods

Distributed training of massive machine learning models, in particular d...
09/27/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Relaxed Schedulers Can Efficiently Parallelize Iterative Algorithms

There has been significant progress in understanding the parallelism inh...
08/13/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Distributionally Linearizable Data Structures

Relaxed concurrent data structures have become increasingly popular, due...
04/03/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
The Transactional Conflict Problem

The transactional conflict problem arises in transactional systems whene...
04/03/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
Byzantine Stochastic Gradient Descent

This paper studies the problem of distributed stochastic optimization in...
03/23/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
The Convergence of Stochastic Gradient Descent in Asynchronous Shared Memory

Stochastic Gradient Descent (SGD) is a fundamental algorithm in machine ...
03/23/2018 ∙ by Dan Alistarh, et al. ∙ 0 ∙ share

read it
SparCML: High-Performance Sparse Communication for Machine Learning

One of the main drivers behind the rapid recent advances in machine lear...
02/22/2018 ∙ by Cèdric Renggli, et al. ∙ 0 ∙ share

read it
Model compression via distillation and quantization

Deep neural networks (DNNs) continue to make significant advances, solvi...
02/15/2018 ∙ by Antonio Polino, et al. ∙ 0 ∙ share

read it
Compressive Sensing with Low Precision Data Representation: Radio Astronomy and Beyond

Modern scientific instruments produce vast amounts of data, which can ov...
02/14/2018 ∙ by Nezihe Merve Gürel, et al. ∙ 0 ∙ share

read it
DataBright: Towards a Global Exchange for Decentralized Data Ownership and Trusted Computation

It is safe to assume that, for the foreseeable future, machine learning,...
02/13/2018 ∙ by David Dao, et al. ∙ 0 ∙ share

read it
The ZipML Framework for Training Models with End-to-End Low Precision: The Cans, the Cannots, and a Little Bit of Deep Learning

Recently there has been significant interest in training machine-learnin...
11/16/2016 ∙ by Hantian Zhang, et al. ∙ 0 ∙ share

read it

Dan Alistarh

Featured Co-authors