Subsampled Rényi Differential Privacy and Analytical Moments Accountant
share
We study the problem of subsampling in differential privacy (DP), a question that is the centerpiece behind many successful differentially private machine learning algorithms. Specifically, we provide a tight upper bound on the Rényi Differential Privacy (RDP) (Mironov, 2017) parameters for algorithms that: (1) subsample the dataset, and then (2) apply a randomized mechanism M to the subsample, in terms of the RDP parameters of M and the subsampling probability parameter. This result generalizes the classic subsampling-based "privacy amplification" property of (ϵ,δ)-differential privacy that applies to only one fixed pair of (ϵ,δ), to a stronger version that exploits properties of each specific randomized algorithm and satisfies an entire family of (ϵ(δ),δ)-differential privacy for all δ∈ [0,1]. Our experiments confirm the advantage of using our techniques over keeping track of (ϵ,δ) directly, especially in the setting where we need to compose many rounds of data access.
READ FULL TEXT