An Asynchronous Distributed Framework for Large-scale Learning Based on Parameter Exchanges

05/22/2017
by   Bikash Joshi, et al.
0

In many distributed learning problems, the heterogeneous loading of computing machines may harm the overall performance of synchronous strategies. In this paper, we propose an effective asynchronous distributed framework for the minimization of a sum of smooth functions, where each machine performs iterations in parallel on its local function and updates a shared parameter asynchronously. In this way, all machines can continuously work even though they do not have the latest version of the shared parameter. We prove the convergence of the consistency of this general distributed asynchronous method for gradient iterations then show its efficiency on the matrix factorization problem for recommender systems and on binary classification.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/22/2020

Asynchronous Distributed Optimization with Randomized Delays

In this work, we study asynchronous finite sum minimization in a distrib...
research
09/04/2020

Asynchronous Richardson iterations

We consider asynchronous versions of the first and second order Richards...
research
07/28/2023

Empirical Study of Straggler Problem in Parameter Server on Iterative Convergent Distributed Machine Learning

The purpose of this study is to test the effectiveness of current stragg...
research
08/17/2017

More Iterations per Second, Same Quality -- Why Asynchronous Algorithms may Drastically Outperform Traditional Ones

In this paper, we consider the convergence of a very general asynchronou...
research
10/16/2017

Maiter: An Asynchronous Graph Processing Framework for Delta-based Accumulative Iterative Computation

Myriad of graph-based algorithms in machine learning and data mining req...
research
09/29/2021

Delayed Asynchronous Iterative Graph Algorithms

Iterative graph algorithms often compute intermediate values and update ...
research
12/03/2020

Dynamic Asynchronous Iterations

Many problems can be solved by iteration by multiple participants (proce...

Please sign up or login with your details

Forgot password? Click here to reset