On the Communication Latency of Wireless Decentralized Learning

02/10/2020
by   Navid NaderiAlizadeh, et al.
0

We consider a wireless network comprising n nodes located within a circular area of radius R, which are participating in a decentralized learning algorithm to optimize a global objective function using their local datasets. To enable gradient exchanges across the network, we assume each node communicates only with a set of neighboring nodes, which are within a distance R n^-β of itself, where β∈(0,1/2). We use tools from network information theory and random geometric graph theory to show that the communication delay for a single round of exchanging gradients on all the links throughout the network scales as O(n^2-3β/βlog n), increasing (at different rates) with both the number of nodes and the gradient exchange threshold distance.

READ FULL TEXT
research
03/06/2020

Decentralized SGD with Over-the-Air Computation

We study the performance of decentralized stochastic gradient descent (D...
research
12/23/2021

Decentralized Multi-Task Stochastic Optimization With Compressed Communications

We consider a multi-agent network where each node has a stochastic (loca...
research
08/01/2022

Event Notifications in Value-Adding Networks

Linkages between research outputs are crucial in the scholarly knowledge...
research
09/28/2022

Neighborhood Gradient Clustering: An Efficient Decentralized Learning Method for Non-IID Data Distributions

Decentralized learning over distributed datasets can have significantly ...
research
09/26/2017

Network Topology and Communication-Computation Tradeoffs in Decentralized Optimization

In decentralized optimization, nodes cooperate to minimize an overall ob...
research
04/27/2021

Communication Costs in a Geometric Communication Network

A communication network is a graph in which each node has only local inf...
research
09/09/2009

Greedy Gossip with Eavesdropping

This paper presents greedy gossip with eavesdropping (GGE), a novel rand...

Please sign up or login with your details

Forgot password? Click here to reset