Implicit Model Specialization through DAG-based Decentralized Federated Learning

11/01/2021
by   Jossekin Beilharz, et al.
0

Federated learning allows a group of distributed clients to train a common machine learning model on private data. The exchange of model updates is managed either by a central entity or in a decentralized way, e.g. by a blockchain. However, the strong generalization across all clients makes these approaches unsuited for non-independent and identically distributed (non-IID) data. We propose a unified approach to decentralization and personalization in federated learning that is based on a directed acyclic graph (DAG) of model updates. Instead of training a single global model, clients specialize on their local data while using the model updates from other clients dependent on the similarity of their respective data. This specialization implicitly emerges from the DAG-based communication and selection of model updates. Thus, we enable the evolution of specialized models, which focus on a subset of the data and therefore cover non-IID data better than federated learning in a centralized or blockchain-based setup. To the best of our knowledge, the proposed solution is the first to unite personalization and poisoning robustness in fully decentralized federated learning. Our evaluation shows that the specialization of models emerges directly from the DAG-based communication of model updates on three different datasets. Furthermore, we show stable model accuracy and less variance across clients when compared to federated averaging.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

12/24/2020

Decentralized Federated Learning via Mutual Knowledge Transfer

In this paper, we investigate the problem of decentralized federated lea...
07/18/2021

Decentralized federated learning of deep neural networks on non-iid data

We tackle the non-convex problem of learning a personalized deep learnin...
09/18/2019

Detailed comparison of communication efficiency of split learning and federated learning

We compare communication efficiencies of two compelling distributed mach...
12/12/2021

Efficient and Reliable Overlay Networks for Decentralized Federated Learning

We propose near-optimal overlay networks based on d-regular expander gra...
11/22/2020

A decentralized aggregation mechanism for training deep learning models using smart contract system for bank loan prediction

Data privacy and sharing has always been a critical issue when trying to...
08/19/2021

Cross-Silo Federated Learning for Multi-Tier Networks with Vertical and Horizontal Data Partitioning

We consider federated learning in tiered communication networks. Our net...
10/15/2021

FedMe: Federated Learning via Model Exchange

Federated learning is a distributed machine learning method in which a s...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.