FedAUXfdp: Differentially Private One-Shot Federated Distillation

05/30/2022
by   Haley Hoech, et al.
0

Federated learning suffers in the case of non-iid local datasets, i.e., when the distributions of the clients' data are heterogeneous. One promising approach to this challenge is the recently proposed method FedAUX, an augmentation of federated distillation with robust results on even highly heterogeneous client data. FedAUX is a partially (ϵ, δ)-differentially private method, insofar as the clients' private data is protected in only part of the training it takes part in. This work contributes a fully differentially private modification, termed FedAUXfdp. We further contribute an upper bound on the l_2-sensitivity of regularized multinomial logistic regression. In experiments with deep networks on large-scale image datasets, FedAUXfdp with strong differential privacy guarantees performs significantly better than other equally privatized SOTA baselines on non-iid client data in just a single communication round. Full privatization of the modified method results in a negligible reduction in accuracy at all levels of data heterogeneity.

READ FULL TEXT
research
11/24/2019

Differentially Private Federated Variational Inference

In many real-world applications of machine learning, data are distribute...
research
06/17/2021

Locally Differentially Private Federated Learning: Efficient Algorithms with Tight Risk Bounds

Federated learning (FL) is a distributed learning paradigm in which many...
research
06/10/2023

Differentially private sliced inverse regression in the federated paradigm

We extend the celebrated sliced inverse regression to address the challe...
research
12/17/2021

Federated Learning with Heterogeneous Data: A Superquantile Optimization Approach

We present a federated learning framework that is designed to robustly d...
research
06/13/2021

DP-NormFedAvg: Normalizing Client Updates for Privacy-Preserving Federated Learning

In this paper, we focus on facilitating differentially private quantized...
research
01/25/2021

Federated Intrusion Detection for IoT with Heterogeneous Cohort Privacy

Internet of Things (IoT) devices are becoming increasingly popular and a...

Please sign up or login with your details

Forgot password? Click here to reset