Analytical Composition of Differential Privacy via the Edgeworth Accountant

06/09/2022
by   Hua Wang, et al.
0

Many modern machine learning algorithms are composed of simple private algorithms; thus, an increasingly important problem is to efficiently compute the overall privacy loss under composition. In this study, we introduce the Edgeworth Accountant, an analytical approach to composing differential privacy guarantees of private algorithms. The Edgeworth Accountant starts by losslessly tracking the privacy loss under composition using the f-differential privacy framework, which allows us to express the privacy guarantees using privacy-loss log-likelihood ratios (PLLRs). As the name suggests, this accountant next uses the Edgeworth expansion to the upper and lower bounds the probability distribution of the sum of the PLLRs. Moreover, by relying on a technique for approximating complex distributions using simple ones, we demonstrate that the Edgeworth Accountant can be applied to the composition of any noise-addition mechanism. Owing to certain appealing features of the Edgeworth expansion, the (ϵ, δ)-differential privacy bounds offered by this accountant are non-asymptotic, with essentially no extra computational cost, as opposed to the prior approaches in, wherein the running times increase with the number of compositions. Finally, we demonstrate that our upper and lower (ϵ, δ)-differential privacy bounds are tight in federated analytics and certain regimes of training private deep learning models.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/10/2020

Sharp Composition Bounds for Gaussian Differential Privacy via Edgeworth Expansion

Datasets containing sensitive information are often sequentially analyze...
research
03/10/2022

Fully Adaptive Composition in Differential Privacy

Composition is a key feature of differential privacy. Well-known advance...
research
09/30/2019

Optimal Differential Privacy Composition for Exponential Mechanisms and the Cost of Adaptivity

Composition is one of the most important properties of differential priv...
research
08/20/2018

Privacy Amplification by Iteration

Many commonly used learning algorithms work by iteratively updating an i...
research
05/31/2023

Concentrated Geo-Privacy

This paper proposes concentrated geo-privacy (CGP), a privacy notion tha...
research
04/25/2023

Differential Privacy via Distributionally Robust Optimization

In recent years, differential privacy has emerged as the de facto standa...
research
02/24/2021

Computing Differential Privacy Guarantees for Heterogeneous Compositions Using FFT

The recently proposed Fast Fourier Transform (FFT)-based accountant for ...

Please sign up or login with your details

Forgot password? Click here to reset