Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes

11/17/2022

∙

Communication costs within Federated learning hinder the system scalability for reaching more data from more clients. The proposed FL adopts a hub-and-spoke network topology. All clients communicate through the central server. Hence, reducing communication overheads via techniques such as data compression has been proposed to mitigate this issue. Another challenge of federated learning is unbalanced data distribution, data on each client are not independent and identically distributed (non-IID) in a typical federated learning setting. In this paper, we proposed a new compression compensation scheme called Global Momentum Fusion (GMF) which reduces communication overheads between FL clients and the server and maintains comparable model accuracy in the presence of non-IID data. GitHub repository: https://github.com/tony92151/global-momentum-fusion-fl

READ FULL TEXT

Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes

Communication-Efficient Federated Learning through Importance Sampling

Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression

A Novel Optimized Asynchronous Federated Learning Framework

Communication-Efficient Federated Learning with Acceleration of Global Momentum

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

Mobilizing Personalized Federated Learning via Random Walk Stochastic ADMM

Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data

Improving Federated Learning Communication Efficiency with Global Momentum Fusion for Gradient Compression Schemes

Related Research

Communication-Efficient Federated Learning through Importance Sampling

Communication-Efficient Federated Learning with Dual-Side Low-Rank Compression

A Novel Optimized Asynchronous Federated Learning Framework

Communication-Efficient Federated Learning with Acceleration of Global Momentum

FedCLIP: Fast Generalization and Personalization for CLIP in Federated Learning

Mobilizing Personalized Federated Learning via Random Walk Stochastic ADMM

Vertical Federated Principal Component Analysis and Its Kernel Extension on Feature-wise Distributed Data