Continual Learning with Differential Privacy

10/11/2021
by   Pradnya Desai, et al.
0

In this paper, we focus on preserving differential privacy (DP) in continual learning (CL), in which we train ML models to learn a sequence of new tasks while memorizing previous tasks. We first introduce a notion of continual adjacent databases to bound the sensitivity of any data record participating in the training process of CL. Based upon that, we develop a new DP-preserving algorithm for CL with a data sampling strategy to quantify the privacy risk of training data in the well-known Averaged Gradient Episodic Memory (A-GEM) approach by applying a moments accountant. Our algorithm provides formal guarantees of privacy for data records across tasks in CL. Preliminary theoretical analysis and evaluations show that our mechanism tightens the privacy loss while maintaining a promising model utility.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/26/2022

Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning

In this paper, we show that the process of continually learning new task...
research
06/30/2022

Fairness and Cost Constrained Privacy-Aware Record Linkage

Record linkage algorithms match and link records from different database...
research
02/21/2022

Personalized PATE: Differential Privacy for Machine Learning with Individual Privacy Guarantees

Applying machine learning (ML) to sensitive domains requires privacy pro...
research
05/19/2022

Differential Privacy: What is all the noise about?

Differential Privacy (DP) is a formal definition of privacy that provide...
research
06/25/2017

Preserving Differential Privacy in Convolutional Deep Belief Networks

The remarkable development of deep learning in medicine and healthcare d...
research
05/07/2021

Differential Privacy for Pairwise Learning: Non-convex Analysis

Pairwise learning focuses on learning tasks with pairwise loss functions...
research
05/26/2023

Incentive Mechanism for Uncertain Tasks under Differential Privacy

Mobile crowd sensing (MCS) has emerged as an increasingly popular sensin...

Please sign up or login with your details

Forgot password? Click here to reset