ARock: an Algorithmic Framework for Asynchronous Parallel Coordinate Updates

by   Zhimin Peng, et al.

Finding a fixed point to a nonexpansive operator, i.e., x^*=Tx^*, abstracts many problems in numerical linear algebra, optimization, and other areas of scientific computing. To solve fixed-point problems, we propose ARock, an algorithmic framework in which multiple agents (machines, processors, or cores) update x in an asynchronous parallel fashion. Asynchrony is crucial to parallel computing since it reduces synchronization wait, relaxes communication bottleneck, and thus speeds up computing significantly. At each step of ARock, an agent updates a randomly selected coordinate x_i based on possibly out-of-date information on x. The agents share x through either global memory or communication. If writing x_i is atomic, the agents can read and write x without memory locks. Theoretically, we show that if the nonexpansive operator T has a fixed point, then with probability one, ARock generates a sequence that converges to a fixed points of T. Our conditions on T and step sizes are weaker than comparable work. Linear convergence is also obtained. We propose special cases of ARock for linear systems, convex optimization, machine learning, as well as distributed and decentralized consensus problems. Numerical experiments of solving sparse logistic regression problems are presented.


page 1

page 2

page 3

page 4


Fast Krasnosel'skii-Mann algorithm with a convergence rate of the fixed point iteration of o(1/k)

The Krasnosel'skii-Mann (KM) algorithm is the most fundamental iterative...

A Distributed Algorithm for Computing a Common Fixed Point of a Finite Family of Paracontractions

A distributed algorithm is described for finding a common fixed point of...

From Local SGD to Local Fixed Point Methods for Federated Learning

Most algorithms for solving optimization problems or finding saddle poin...

Coordinate Friendly Structures, Algorithms and Applications

This paper focuses on coordinate update methods, which are useful for so...

Bargaining for Revenue Shares on Tree Trading Networks

We study trade networks with a tree structure, where a seller with a sin...

Distributed asynchronous convergence detection without detection protocol

In this paper, we address the problem of detecting the moment when an on...

Optimization-Derived Learning with Essential Convergence Analysis of Training and Hyper-training

Recently, Optimization-Derived Learning (ODL) has attracted attention fr...

Please sign up or login with your details

Forgot password? Click here to reset