
Approximate Byzantine FaultTolerance in Distributed Optimization
We consider the problem of Byzantine faulttolerance in distributed mult...
read it

A Canonical Form for FirstOrder Distributed Optimization Algorithms
We consider the distributed optimization problem in which a network of a...
read it

Optimization in Open Networks via Dual Averaging
In networks of autonomous agents (e.g., fleets of vehicles, scattered se...
read it

Collective Online Learning via Decentralized Gaussian Processes in Massive MultiAgent Systems
Distributed machine learning (ML) is a modern computation paradigm that ...
read it

Asynchronous Distributed Optimization with Randomized Delays
In this work, we study asynchronous finite sum minimization in a distrib...
read it

DSPG: Decentralized Simultaneous Perturbations Gradient Descent Scheme
In this paper, we present an asynchronous approximate gradient method th...
read it

The distributed dual ascent algorithm is robust to asynchrony
The distributed dual ascent is an established algorithm to solve strongl...
read it
Asynchronous Distributed Optimization with Redundancy in Cost Functions
This paper considers the problem of asynchronous distributed multiagent optimization on serverbased system architecture. In this problem, each agent has a local cost, and the goal for the agents is to collectively find a minimum of their aggregate cost. A standard algorithm to solve this problem is the iterative distributed gradientdescent (DGD) method being implemented collaboratively by the server and the agents. In the synchronous setting, the algorithm proceeds from one iteration to the next only after all the agents complete their expected communication with the server. However, such synchrony can be expensive and even infeasible in realworld applications. We show that waiting for all the agents is unnecessary in many applications of distributed optimization, including distributed machine learning, due to redundancy in the cost functions (or data). Specifically, we consider a generic notion of redundancy named (r,ϵ)redundancy implying solvability of the original multiagent optimization problem with ϵ accuracy, despite the removal of up to r (out of total n) agents from the system. We present an asynchronous DGD algorithm where in each iteration the server only waits for (any) nr agents, instead of all the n agents. Assuming (r,ϵ)redundancy, we show that our asynchronous algorithm converges to an approximate solution with error that is linear in ϵ and r. Moreover, we also present a generalization of our algorithm to tolerate some Byzantine faulty agents in the system. Finally, we demonstrate the improved communication efficiency of our algorithm through experiments on MNIST and FashionMNIST using the benchmark neural network LeNet.
READ FULL TEXT
Comments
There are no comments yet.