DeepAI AI Chat
Log In Sign Up

Architecture Agnostic Federated Learning for Neural Networks

02/15/2022
by   Disha Makhija, et al.
0

With growing concerns regarding data privacy and rapid increase in data volume, Federated Learning(FL) has become an important learning paradigm. However, jointly learning a deep neural network model in a FL setting proves to be a non-trivial task because of the complexities associated with the neural networks, such as varied architectures across clients, permutation invariance of the neurons, and presence of non-linear transformations in each layer. This work introduces a novel Federated Heterogeneous Neural Networks (FedHeNN) framework that allows each client to build a personalised model without enforcing a common architecture across clients. This allows each client to optimize with respect to local data and compute constraints, while still benefiting from the learnings of other (potentially more powerful) clients. The key idea of FedHeNN is to use the instance-level representations obtained from peer clients to guide the simultaneous training on each client. The extensive experimental results demonstrate that the FedHeNN framework is capable of learning better performing models on clients in both the settings of homogeneous and heterogeneous architectures across clients.

READ FULL TEXT
06/13/2023

Privacy Preserving Bayesian Federated Learning in Heterogeneous Settings

In several practical applications of federated learning (FL), the client...
05/25/2022

Federated Self-supervised Learning for Heterogeneous Clients

Federated Learning has become an important learning paradigm due to its ...
03/28/2022

Federated Learning with Position-Aware Neurons

Federated Learning (FL) fuses collaborative models from local nodes with...
01/20/2022

Federated Learning with Heterogeneous Architectures using Graph HyperNetworks

Standard Federated Learning (FL) techniques are limited to clients with ...
12/27/2021

SPIDER: Searching Personalized Neural Architecture for Federated Learning

Federated learning (FL) is an efficient learning framework that assists ...
06/05/2023

Unlocking the Potential of Federated Learning for Deeper Models

Federated learning (FL) is a new paradigm for distributed machine learni...
02/04/2021

FedAUX: Leveraging Unlabeled Auxiliary Data in Federated Learning

Federated Distillation (FD) is a popular novel algorithmic paradigm for ...