Benchmarking Semi-supervised Federated Learning

08/26/2020
by   Zhengming Zhang, et al.
15

Federated learning promises to use the computational power of edge devices while maintaining user data privacy. Current frameworks, however, typically make the unrealistic assumption that the data stored on user devices come with ground truth labels, while the server has no data. In this work, we consider the more realistic scenario where the users have only unlabeled data and the server has a limited amount of labeled data. In this semi-supervised federated learning (ssfl) setting, the data distribution can be non-iid, in the sense of different distributions of classes at different users. We define a metric, R, to measure this non-iidness in class distributions. In this setting, we provide a thorough study on different factors that can affect the final test accuracy, including algorithm design (such as training objective), the non-iidness R, the communication period T, the number of users K, the amount of labeled data in the server N_s, and the number of users C_k≤ K that communicate with the server in each communication round. We evaluate our ssfl framework on Cifar-10, SVHN, and EMNIST. Overall, we find that a simple consistency loss-based method, along with group normalization, achieves better generalization performance, even compared to previous supervised federated learning settings. Furthermore, we propose a novel grouping-based model average method to improve convergence efficiency, and we show that this can boost performance by up to 10.79 method.

READ FULL TEXT

page 6

page 8

page 9

page 10

page 15

page 16

page 17

page 18

06/02/2021

SemiFL: Communication Efficient Semi-Supervised Federated Learning with Unlabeled Clients

Federated Learning allows training machine learning models by using the ...
03/15/2022

SemiPFL: Personalized Semi-Supervised Federated Learning Framework for Edge Intelligence

Recent advances in wearable devices and Internet-of-Things (IoT) have le...
08/21/2021

SemiFed: Semi-supervised Federated Learning with Consistency and Pseudo-Labeling

Federated learning enables multiple clients, such as mobile phones and o...
01/10/2020

Exploiting Unlabeled Data in Smart Cities using Federated Learning

Privacy concerns are considered one of the main challenges in smart citi...
09/13/2019

Measuring the Effects of Non-Identical Data Distribution for Federated Visual Classification

Federated Learning enables visual models to be trained in a privacy-pres...
04/21/2020

Federated Learning with Only Positive Labels

We consider learning a multi-class classification model in the federated...
06/28/2021

Chat Room Using HTML, PHP, CSS, JS, AJAX

Earlier there was no mode of online communication between users. In big ...

Code Repositories

SSFL-Benchmarking-Semi-supervised-Federated-Learning

Benchmarking Semi-supervised Federated Learning


view repo