Federated Learning with Privacy-Preserving Ensemble Attention Distillation

10/16/2022
by   Xuan Gong, et al.
0

Federated Learning (FL) is a machine learning paradigm where many local nodes collaboratively train a central model while keeping the training data decentralized. This is particularly relevant for clinical applications since patient data are usually not allowed to be transferred out of medical facilities, leading to the need for FL. Existing FL methods typically share model parameters or employ co-distillation to address the issue of unbalanced data distribution. However, they also require numerous rounds of synchronized communication and, more importantly, suffer from a privacy leakage risk. We propose a privacy-preserving FL framework leveraging unlabeled public data for one-way offline knowledge distillation in this work. The central model is learned from local knowledge via ensemble attention distillation. Our technique uses decentralized and heterogeneous local data like existing FL approaches, but more importantly, it significantly reduces the risk of privacy leakage. We demonstrate that our method achieves very competitive performance with more robust privacy preservation based on extensive experiments on image classification, segmentation, and reconstruction tasks.

READ FULL TEXT

page 1

page 3

page 5

page 8

research
09/10/2022

Preserving Privacy in Federated Learning with Ensemble Cross-Domain Knowledge Distillation

Federated Learning (FL) is a machine learning paradigm where local nodes...
research
06/12/2020

Ensemble Distillation for Robust Model Fusion in Federated Learning

Federated Learning (FL) is a machine learning setting where many devices...
research
06/20/2022

Decentralized Distributed Learning with Privacy-Preserving Data Synthesis

In the medical field, multi-center collaborations are often sought to yi...
research
04/03/2021

Knowledge Distillation For Wireless Edge Learning

In this paper, we propose a framework for predicting frame errors in the...
research
05/09/2023

FedPDD: A Privacy-preserving Double Distillation Framework for Cross-silo Federated Recommendation

Cross-platform recommendation aims to improve recommendation accuracy by...
research
06/19/2023

FSAR: Federated Skeleton-based Action Recognition with Adaptive Topology Structure and Knowledge Distillation

Existing skeleton-based action recognition methods typically follow a ce...
research
10/28/2022

Completely Heterogeneous Federated Learning

Federated learning (FL) faces three major difficulties: cross-domain, he...

Please sign up or login with your details

Forgot password? Click here to reset