In-network Computation for Large-scale Federated Learning over Wireless Edge Networks

09/22/2021
by   Thinh Quang Dinh, et al.
0

Most conventional Federated Learning (FL) models are using a star network topology where all users aggregate their local models at a single server (e.g., a cloud server). That causes significant overhead in terms of both communications and computing at the server, delaying the training process, especially for large scale FL systems with straggling nodes. This paper proposes a novel edge network architecture that enables decentralizing the model aggregation process at the server, thereby significantly reducing the training delay for the whole FL network. Specifically, we design a highly-effective in-network computation protocol (INC) consisting of a user scheduling mechanism, an in-network aggregation process (INA) which is designed for both primal- and primal-dual methods in distributed machine learning problems, and a network routing algorithm. Under the proposed INA, we then formulate a joint routing and resource optimization problem, aiming to minimize the aggregation latency. The problem is NP-hard, and thus we propose a polynomial time routing algorithm which can achieve near optimal performance with a theoretical bound. Simulation results showed that the proposed INC framework can not only help reduce the FL training latency, up to 5.6 times, but also significantly decrease cloud's traffic and computing overhead. This can enable large-scale FL.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/22/2021

Enabling Large-Scale Federated Learning over Wireless Edge Networks

Major bottlenecks of large-scale Federated Learning(FL) networks are the...
research
08/20/2022

Just-in-Time Aggregation for Federated Learning

The increasing number and scale of federated learning (FL) jobs necessit...
research
10/22/2020

Hierarchical Federated Learning through LAN-WAN Orchestration

Federated learning (FL) was designed to enable mobile phones to collabor...
research
04/15/2022

Server Free Wireless Federated Learning: Architecture, Algorithm, and Analysis

We demonstrate that merely analog transmissions and match filtering can ...
research
12/15/2021

Blockchain-enabled Server-less Federated Learning

Motivated by the heterogeneous nature of devices participating in large-...
research
08/05/2021

User Scheduling for Federated Learning Through Over-the-Air Computation

A new machine learning (ML) technique termed as federated learning (FL) ...
research
02/05/2021

Learning Rate Optimization for Federated Learning Exploiting Over-the-air Computation

Federated learning (FL) as a promising edge-learning framework can effec...

Please sign up or login with your details

Forgot password? Click here to reset