Exploiting Features and Logits in Heterogeneous Federated Learning

by   Yun-Hin Chan, et al.

Due to the rapid growth of IoT and artificial intelligence, deploying neural networks on IoT devices is becoming increasingly crucial for edge intelligence. Federated learning (FL) facilitates the management of edge devices to collaboratively train a shared model while maintaining training data local and private. However, a general assumption in FL is that all edge devices are trained on the same machine learning model, which may be impractical considering diverse device capabilities. For instance, less capable devices may slow down the updating process because they struggle to handle large models appropriate for ordinary devices. In this paper, we propose a novel data-free FL method that supports heterogeneous client models by managing features and logits, called Felo; and its extension with a conditional VAE deployed in the server, called Velo. Felo averages the mid-level features and logits from the clients at the server based on their class labels to provide the average features and logits, which are utilized for further training the client models. Unlike Felo, the server has a conditional VAE in Velo, which is used for training mid-level features and generating synthetic features according to the labels. The clients optimize their models based on the synthetic features and the average logits. We conduct experiments on two datasets and show satisfactory performances of our methods compared with the state-of-the-art methods.


page 1

page 6


HeteroFL: Computation and Communication Efficient Federated Learning for Heterogeneous Clients

Federated Learning (FL) is a method of training machine learning models ...

Heterogeneous Ensemble Knowledge Transfer for Training Large Models in Federated Learning

Federated learning (FL) enables edge-devices to collaboratively learn a ...

FedIN: Federated Intermediate Layers Learning for Model Heterogeneity

Federated learning (FL) facilitates edge devices to cooperatively train ...

FedAR: Activity and Resource-Aware Federated Learning Model for Distributed Mobile Robots

Smartphones, autonomous vehicles, and the Internet-of-things (IoT) devic...

Multimodal Multiple Federated Feature Construction Method for IoT Environments

The fast development of Internet-of-Things (IoT) devices and application...

EdgeConvEns: Convolutional Ensemble Learning for Edge Intelligence

Deep edge intelligence aims to deploy deep learning models that demand c...

FedCache: A Knowledge Cache-driven Federated Learning Architecture for Personalized Edge Intelligence

Edge Intelligence (EI) allows Artificial Intelligence (AI) applications ...

Please sign up or login with your details

Forgot password? Click here to reset