Federated Semi-Supervised Learning with Class Distribution Mismatch

by   Zhiguo Wang, et al.

Many existing federated learning (FL) algorithms are designed for supervised learning tasks, assuming that the local data owned by the clients are well labeled. However, in many practical situations, it could be difficult and expensive to acquire complete data labels. Federated semi-supervised learning (Fed-SSL) is an attractive solution for fully utilizing both labeled and unlabeled data. Similar to that encountered in federated supervised learning, class distribution of labeled/unlabeled data could be non-i.i.d. among clients. Besides, in each client, the class distribution of labeled data may be distinct from that of unlabeled data. Unfortunately, both can severely jeopardize the FL performance. To address such challenging issues, we introduce two proper regularization terms that can effectively alleviate the class distribution mismatch problem in Fed-SSL. In addition, to overcome the non-i.i.d. data, we leverage the variance reduction and normalized averaging techniques to develop a novel Fed-SSL algorithm. Theoretically, we prove that the proposed method has a convergence rate of 𝒪(1/√(T)), where T is the number of communication rounds, even when the data distribution are non-i.i.d. among clients. To the best of our knowledge, it is the first formal convergence result for Fed-SSL problems. Numerical experiments based on MNIST data and CIFAR-10 data show that the proposed method can greatly improve the classification accuracy compared to baselines.


page 3

page 4

page 5

page 6

page 7

page 8

page 9

page 14


Combating Data Imbalances in Federated Semi-supervised Learning with Dual Regulators

Federated learning has become a popular method to learn from decentraliz...

Local or Global: Selective Knowledge Assimilation for Federated Learning with Limited Labels

Many existing FL methods assume clients with fully-labeled data, while i...

Federated Semi-Supervised Learning with Prototypical Networks

With the increasing computing power of edge devices, Federated Learning ...

Towards Unbiased Training in Federated Open-world Semi-supervised Learning

Federated Semi-supervised Learning (FedSSL) has emerged as a new paradig...

Uncertainty Minimization for Personalized Federated Semi-Supervised Learning

Since federated learning (FL) has been introduced as a decentralized lea...

Federated Learning with Positive and Unlabeled Data

We study the problem of learning from positive and unlabeled (PU) data i...

Federated Semi-Supervised Learning with Annotation Heterogeneity

Federated Semi-Supervised Learning (FSSL) aims to learn a global model f...

Please sign up or login with your details

Forgot password? Click here to reset