Federated Learning over Wireless Networks: A Band-limited Coordinated Descent Approach

02/16/2021
by   Junshan Zhang, et al.
0

We consider a many-to-one wireless architecture for federated learning at the network edge, where multiple edge devices collaboratively train a model using local data. The unreliable nature of wireless connectivity, together with constraints in computing resources at edge devices, dictates that the local updates at edge devices should be carefully crafted and compressed to match the wireless communication resources available and should work in concert with the receiver. Thus motivated, we propose SGD-based bandlimited coordinate descent algorithms for such settings. Specifically, for the wireless edge employing over-the-air computing, a common subset of k-coordinates of the gradient updates across edge devices are selected by the receiver in each iteration, and then transmitted simultaneously over k sub-carriers, each experiencing time-varying channel conditions. We characterize the impact of communication error and compression, in terms of the resulting gradient bias and mean squared error, on the convergence of the proposed algorithms. We then study learning-driven communication error minimization via joint optimization of power allocation and learning rates. Our findings reveal that optimal power allocation across different sub-carriers should take into account both the gradient values and channel conditions, thus generalizing the widely used water-filling policy. We also develop sub-optimal distributed solutions amenable to implementation.

READ FULL TEXT
research
07/23/2019

Federated Learning over Wireless Fading Channels

We study federated machine learning at the wireless network edge, where ...
research
01/28/2020

Update Aware Device Scheduling for Federated Learning at the Wireless Edge

We study federated learning (FL) at the wireless edge, where power-limit...
research
07/21/2020

Learning Centric Power Allocation for Edge Intelligence

While machine-type communication (MTC) devices generate massive data, th...
research
01/03/2019

Machine Learning at the Wireless Edge: Distributed Stochastic Gradient Descent Over-the-Air

We study collaborative machine learning at the wireless edge, where powe...
research
05/08/2022

Over-the-Air Federated Multi-Task Learning via Model Sparsification and Turbo Compressed Sensing

To achieve communication-efficient federated multitask learning (FMTL), ...
research
12/07/2021

Collaborative Learning over Wireless Networks: An Introductory Overview

In this chapter, we will mainly focus on collaborative training across w...
research
11/07/2022

How to Coordinate Edge Devices for Over-the-Air Federated Learning?

This work studies the task of device coordination in wireless networks f...

Please sign up or login with your details

Forgot password? Click here to reset