Lifelong DP: Consistently Bounded Differential Privacy in Lifelong Machine Learning

07/26/2022
by   Phung Lai, et al.
1

In this paper, we show that the process of continually learning new tasks and memorizing previous tasks introduces unknown privacy risks and challenges to bound the privacy loss. Based upon this, we introduce a formal definition of Lifelong DP, in which the participation of any data tuples in the training set of any tasks is protected, under a consistently bounded DP protection, given a growing stream of tasks. A consistently bounded DP means having only one fixed value of the DP privacy budget, regardless of the number of tasks. To preserve Lifelong DP, we propose a scalable and heterogeneous algorithm, called L2DP-ML with a streaming batch training, to efficiently train and continue releasing new versions of an L2M model, given the heterogeneity in terms of data sizes and the training order of tasks, without affecting DP protection of the private training set. An end-to-end theoretical analysis and thorough evaluations show that our mechanism is significantly better than baseline approaches in preserving Lifelong DP. The implementation of L2DP-ML is available at: https://github.com/haiphanNJIT/PrivateDeepLearning.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
12/26/2022

Packing Privacy Budget Efficiently

Machine learning (ML) models can leak information about users, and diffe...
research
10/11/2021

Continual Learning with Differential Privacy

In this paper, we focus on preserving differential privacy (DP) in conti...
research
10/25/2021

DP-XGBoost: Private Machine Learning at Scale

The big-data revolution announced ten years ago does not seem to have fu...
research
03/01/2023

How to DP-fy ML: A Practical Guide to Machine Learning with Differential Privacy

ML models are ubiquitous in real world applications and are a constant f...
research
03/29/2020

Dealer: End-to-End Data Marketplace with Model-based Pricing

Data-driven machine learning (ML) has witnessed great successes across a...
research
02/20/2023

Efficient Privacy-Preserved Processing of Multimodal Data for Vehicular Traffic Analysis

We estimate vehicular traffic states from multimodal data collected by s...
research
03/16/2022

Differentiable DAG Sampling

We propose a new differentiable probabilistic model over DAGs (DP-DAG). ...

Please sign up or login with your details

Forgot password? Click here to reset