Decentralized SGD with Over-the-Air Computation

03/06/2020
by   Emre Ozfatura, et al.
6

We study the performance of decentralized stochastic gradient descent (DSGD) in a wireless network, where the nodes collaboratively optimize an objective function using their local datasets. Unlike the conventional setting, where the nodes communicate over error-free orthogonal communication links, we assume that transmissions are prone to additive noise and interference.We first consider a point-to-point (P2P) transmission strategy, termed the OAC-P2P scheme, in which the node pairs are scheduled in an orthogonal fashion to minimize interference. Since in the DSGD framework, each node requires a linear combination of the neighboring models at the consensus step, we then propose the OAC-MAC scheme, which utilizes the signal superposition property of the wireless medium to achieve over-the-air computation (OAC). For both schemes, we cast the scheduling problem as a graph coloring problem. We numerically evaluate the performance of these two schemes for the MNIST image classification task under various network conditions. We show that the OAC-MAC scheme attains better convergence performance with a fewer communication rounds.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/19/2022

Non-Coherent Over-the-Air Decentralized Stochastic Gradient Descent

This paper proposes a Decentralized Stochastic Gradient Descent (DSGD) a...
research
02/10/2020

On the Communication Latency of Wireless Decentralized Learning

We consider a wireless network comprising n nodes located within a circu...
research
07/26/2021

Decentralized Federated Learning: Balancing Communication and Computing Costs

Decentralized federated learning (DFL) is a powerful framework of distri...
research
02/28/2020

Decentralized Federated Learning via SGD over Wireless D2D Networks

Federated Learning (FL), an emerging paradigm for fast intelligent acqui...
research
02/25/2020

Network-Density-Controlled Decentralized Parallel Stochastic Gradient Descent in Wireless Systems

This paper proposes a communication strategy for decentralized learning ...
research
12/23/2021

Decentralized Multi-Task Stochastic Optimization With Compressed Communications

We consider a multi-agent network where each node has a stochastic (loca...
research
05/04/2018

Scheduling wireless links in the physical interference model by fractional edge coloring

We consider the problem of scheduling the links of wireless mesh network...

Please sign up or login with your details

Forgot password? Click here to reset