The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset

08/07/2021

∙

Federated Learning is an algorithm suited for training models on decentralized data, but the requirement of a central "server" node is a bottleneck. In this document, we first introduce the notion of Decentralized Federated Learning (DFL). We then perform various experiments on different setups, such as changing model aggregation frequency, switching from independent and identically distributed (IID) dataset partitioning to non-IID partitioning with partial global sharing, using different optimization methods across clients, and breaking models into segments with partial sharing. All experiments are run on the MNIST handwritten digits dataset. We observe that those altered training procedures are generally robust, albeit non-optimal. We also observe failures in training when the variance between model weights is too large. The open-source experiment code is accessible through GitHub[Code was uploaded at <https://github.com/zhzhang2018/DecentralizedFL>].

READ FULL TEXT

The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset

Decentralized Federated Learning via Mutual Knowledge Transfer

Scatterbrained: A flexible and expandable pattern for decentralized machine learning

Blades: A Simulator for Attacks and Defenses in Federated Learning

Provably Secure Federated Learning against Malicious Clients

FedOS: using open-set learning to stabilize training in federated learning

Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees

On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing

The Effect of Training Parameters and Mechanisms on Decentralized Federated Learning based on MNIST Dataset

Related Research

Decentralized Federated Learning via Mutual Knowledge Transfer

Scatterbrained: A flexible and expandable pattern for decentralized machine learning

Blades: A Simulator for Attacks and Defenses in Federated Learning

Provably Secure Federated Learning against Malicious Clients

FedOS: using open-set learning to stabilize training in federated learning

Byzantine-Robust Federated Learning with Optimal Statistical Rates and Privacy Guarantees

On the Convergence of Decentralized Federated Learning Under Imperfect Information Sharing