Towards Federated Learning on Time-Evolving Heterogeneous Data

12/25/2021
by   Yongxin Guo, et al.
0

Federated Learning (FL) is an emerging learning paradigm that preserves privacy by ensuring client data locality on edge devices. The optimization of FL is challenging in practice due to the diversity and heterogeneity of the learning system. Despite recent research efforts on improving the optimization of heterogeneous data, the impact of time-evolving heterogeneous data in real-world scenarios, such as changing client data or intermittent clients joining or leaving during training, has not been well studied. In this work, we propose Continual Federated Learning (CFL), a flexible framework, to capture the time-evolving heterogeneity of FL. CFL covers complex and realistic scenarios – which are challenging to evaluate in previous FL formulations – by extracting the information of past local datasets and approximating the local objective functions. Theoretically, we demonstrate that CFL methods achieve a faster convergence rate than in time-evolving scenarios, with the benefit being dependent on approximation quality. In a series of experiments, we show that the numerical findings match the convergence analysis, and CFL methods significantly outperform the other SOTA FL baselines.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/07/2022

Closing the Gap between Client and Global Model Performance in Heterogeneous Federated Learning

The heterogeneity of hardware and data is a well-known and studied probl...
research
12/19/2022

Adaptive Control of Client Selection and Gradient Compression for Efficient Federated Learning

Federated learning (FL) allows multiple clients cooperatively train mode...
research
06/14/2022

Disentangled Federated Learning for Tackling Attributes Skew via Invariant Aggregation and Diversity Transferring

Attributes skew hinders the current federated learning (FL) frameworks f...
research
01/27/2022

On the Convergence of Heterogeneous Federated Learning with Arbitrary Adaptive Online Model Pruning

One of the biggest challenges in Federated Learning (FL) is that client ...
research
10/28/2022

Completely Heterogeneous Federated Learning

Federated learning (FL) faces three major difficulties: cross-domain, he...
research
05/30/2022

Towards Fair Federated Recommendation Learning: Characterizing the Inter-Dependence of System and Data Heterogeneity

Federated learning (FL) is an effective mechanism for data privacy in re...
research
06/10/2022

On Convergence of FedProx: Local Dissimilarity Invariant Bounds, Non-smoothness and Beyond

The FedProx algorithm is a simple yet powerful distributed proximal poin...

Please sign up or login with your details

Forgot password? Click here to reset