DeepAI AI Chat
Log In Sign Up

L-DQN: An Asynchronous Limited-Memory Distributed Quasi-Newton Method

by   Bugra Can, et al.
Rutgers University

This work proposes a distributed algorithm for solving empirical risk minimization problems, called L-DQN, under the master/worker communication model. L-DQN is a distributed limited-memory quasi-Newton method that supports asynchronous computations among the worker nodes. Our method is efficient both in terms of storage and communication costs, i.e., in every iteration the master node and workers communicate vectors of size O(d), where d is the dimension of the decision variable, and the amount of memory required on each node is O(md), where m is an adjustable parameter. To our knowledge, this is the first distributed quasi-Newton method with provable global linear convergence guarantees in the asynchronous setting where delays between nodes are present. Numerical experiments are provided to illustrate the theory and the practical performance of our method.


page 1

page 2

page 3

page 4


A Distributed Quasi-Newton Algorithm for Empirical Risk Minimization with Nonsmooth Regularization

In this paper, we propose a communication- and computation- efficient di...

Scaling Up Quasi-Newton Algorithms: Communication Efficient Distributed SR1

In this paper, we present a scalable distributed implementation of the s...

Asynchronous Policy Evaluation in Distributed Reinforcement Learning over Networks

This paper proposes a fully asynchronous scheme for policy evaluation of...

Asynchronous Distributed Learning with Sparse Communications and Identification

In this paper, we present an asynchronous optimization algorithm for dis...

GIANT: Globally Improved Approximate Newton Method for Distributed Optimization

For distributed computing environments, we consider the canonical machin...

LocalNewton: Reducing Communication Bottleneck for Distributed Learning

To address the communication bottleneck problem in distributed optimizat...

Asynchronous Bayesian Learning over a Network

We present a practical asynchronous data fusion model for networked agen...