DACFL: Dynamic Average Consensus Based Federated Learning in Decentralized Topology

by   Zhikun Chen, et al.

Federated learning (FL) is a burgeoning distributed machine learning framework where a central parameter server (PS) coordinates many local users to train a globally consistent model. Conventional federated learning inevitably relies on a centralized topology with a PS. As a result, it will paralyze once the PS fails. To alleviate such a single point failure, especially on the PS, some existing work has provided decentralized FL (DFL) implementations like CDSGD and D-PSGD to facilitate FL in a decentralized topology. However, there are still some problems with these methods, e.g., significant divergence between users' final models in CDSGD and a network-wide model average necessity in D-PSGD. In order to solve these deficiency, this paper devises a new DFL implementation coined as DACFL, where each user trains its model using its own training data and exchanges the intermediate models with its neighbors through a symmetric and doubly stochastic matrix. The DACFL treats the progress of each user's local training as a discrete-time process and employs a first order dynamic average consensus (FODAC) method to track the average model in the absence of the PS. In this paper, we also provide a theoretical convergence analysis of DACFL on the premise of i.i.d data to strengthen its rationality. The experimental results on MNIST, Fashion-MNIST and CIFAR-10 validate the feasibility of our solution in both time-invariant and time-varying network topologies, and declare that DACFL outperforms D-PSGD and CDSGD in most cases.


page 1

page 2

page 3

page 4


Fedstellar: A Platform for Decentralized Federated Learning

In 2016, Google proposed Federated Learning (FL) as a novel paradigm to ...

Robust Blockchained Federated Learning with Model Validation and Proof-of-Stake Inspired Consensus

Federated learning (FL) is a promising distributed learning solution tha...

Enhanced Decentralized Federated Learning based on Consensus in Connected Vehicles

Advanced researches on connected vehicles have recently targeted to the ...

Improving the Model Consistency of Decentralized Federated Learning

To mitigate the privacy leakages and communication burdens of Federated ...

Decentralized Event-Triggered Federated Learning with Heterogeneous Communication Thresholds

A recent emphasis of distributed learning research has been on federated...

Task-load-Aware Game-Theoretic Framework for Wireless Federated Learning

Federated learning (FL) has been proposed as a popular learning framewor...

Forgettable Federated Linear Learning with Certified Data Removal

Federated learning (FL) is a trending distributed learning framework tha...

Please sign up or login with your details

Forgot password? Click here to reset