Federated Learning with Randomized Douglas-Rachford Splitting Methods

03/05/2021
by   Nhan H. Pham, et al.
0

In this paper, we develop two new algorithms, called, FedDR and asyncFedDR, for solving a fundamental nonconvex optimization problem in federated learning. Our algorithms rely on a novel combination between a nonconvex Douglas-Rachford splitting method, randomized block-coordinate strategies, and asynchronous implementation. Unlike recent methods in the literature, e.g., FedSplit and FedPD, our algorithms update only a subset of users at each communication round, and possibly in an asynchronous mode, making them more practical. These new algorithms also achieve communication efficiency and more importantly can handle statistical and system heterogeneity, which are the two main challenges in federated learning. Our convergence analysis shows that the new algorithms match the communication complexity lower bound up to a constant factor under standard assumptions. Our numerical experiments illustrate the advantages of the proposed methods compared to existing ones using both synthetic and real datasets.

READ FULL TEXT
research
08/12/2021

An Operator Splitting View of Federated Learning

Over the past few years, the federated learning () community has witness...
research
05/29/2021

A Federated Learning Framework for Nonconvex-PL Minimax Problems

We consider a general class of nonconvex-PL minimax problems in the cros...
research
11/27/2021

Resource-Aware Asynchronous Online Federated Learning for Nonlinear Regression

Many assumptions in the federated learning literature present a best-cas...
research
11/18/2021

A Novel Optimized Asynchronous Federated Learning Framework

Federated Learning (FL) since proposed has been applied in many fields, ...
research
02/05/2021

Bias-Variance Reduced Local SGD for Less Heterogeneous Federated Learning

Federated learning is one of the important learning scenarios in distrib...
research
11/08/2022

Clustered Federated Learning based on Nonconvex Pairwise Fusion

This study investigates clustered federated learning (FL), one of the fo...
research
04/08/2022

Federated Learning with Partial Model Personalization

We consider two federated learning algorithms for training partially per...

Please sign up or login with your details

Forgot password? Click here to reset