Federated Unsupervised Representation Learning

10/18/2020
by   Fengda Zhang, et al.
0

To leverage enormous unlabeled data on distributed edge devices, we formulate a new problem in federated learning called Federated Unsupervised Representation Learning (FURL) to learn a common representation model without supervision while preserving data privacy. FURL poses two new challenges: (1) data distribution shift (Non-IID distribution) among clients would make local models focus on different categories, leading to the inconsistency of representation spaces. (2) without the unified information among clients in FURL, the representations across clients would be misaligned. To address these challenges, we propose Federated Constrastive Averaging with dictionary and alignment (FedCA) algorithm. FedCA is composed of two key modules: (1) dictionary module to aggregate the representations of samples from each client and share with all clients for consistency of representation space and (2) alignment module to align the representation of each client on a base model trained on a public data. We adopt the contrastive loss for local model training. Through extensive experiments with three evaluation protocols in IID and Non-IID settings, we demonstrate that FedCA outperforms all baselines with significant margins.

READ FULL TEXT

Authors

page 1

11/21/2021

Distributed Unsupervised Visual Representation Learning with Fused Features

Federated learning (FL) enables distributed clients to learn a shared mo...
11/16/2021

Inference-Time Personalized Federated Learning

In Federated learning (FL), multiple clients collaborate to learn a mode...
01/07/2022

Multi-Model Federated Learning

Federated learning is a form of distributed learning with the key challe...
08/14/2021

Collaborative Unsupervised Visual Representation Learning from Decentralized Data

Unsupervised representation learning has achieved outstanding performanc...
04/20/2022

Is Non-IID Data a Threat in Federated Online Learning to Rank?

In this perspective paper we study the effect of non independent and ide...
05/25/2022

Federated Self-supervised Learning for Heterogeneous Clients

Federated Learning has become an important learning paradigm due to its ...
04/01/2021

Federated Few-Shot Learning with Adversarial Learning

We are interested in developing a unified machine learning model over ma...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.