Sparsified Privacy-Masking for Communication-Efficient and Privacy-Preserving Federated Learning

08/01/2020
by   Rui Hu, et al.
17

Federated learning has received significant interests recently due to its capability of learning a shared machine learning model across smart devices without accessing their private data in the era of Internet of things. This paper jointly considers two critical issues of federated learning – data privacy and communication efficiency – and develops a communication-efficient and differentially-private federated learning scheme called CPFed. The main challenge in addressing both issues together lies in the fact that data compression techniques often lead to an increased number of training iterations required for achieving some desired training loss due to the compression errors, while the differential privacy guarantee usually deteriorates with respect to the number of training iterations. To reconcile this dilemma, we propose to use sparsified privacy-masking that first adds random noise to the model update and then applies unbiased random sparsifier before uploading the model update at each device in federated learning. By using sparsified privacy-masking, our proposed CPFed scheme can achieve high communication efficiency and strong data privacy guarantee at the same time while preserving model accuracy. We provide an explict end-to-end privacy guarantee of CPFed using zero-concentrated differential privacy and give its theoretical convergence rates for both convex and non-convex models. Through extensive numerical experiments on real-world datasets, we demonstrate the effectiveness and efficiency of our proposed method.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/30/2020

CPFed: Communication-Efficient and Privacy-Preserving Federated Learning

Federated learning is a machine learning setting where a set of edge dev...
research
09/14/2023

Communication Efficient Private Federated Learning Using Dithering

The task of preserving privacy while ensuring efficient communication is...
research
07/20/2023

Private Federated Learning with Autotuned Compression

We propose new techniques for reducing communication in private federate...
research
03/28/2020

Differentially Private Federated Learning for Resource-Constrained Internet of Things

With the proliferation of smart devices having built-in sensors, Interne...
research
02/08/2022

Practical Challenges in Differentially-Private Federated Survival Analysis of Medical Data

Survival analysis or time-to-event analysis aims to model and predict th...
research
09/19/2023

FRAMU: Attention-based Machine Unlearning using Federated Reinforcement Learning

Machine Unlearning is an emerging field that addresses data privacy issu...
research
07/15/2019

Multi-hop Federated Private Data Augmentation with Sample Compression

On-device machine learning (ML) has brought about the accessibility to a...

Please sign up or login with your details

Forgot password? Click here to reset