RényiTester: A Variational Approach to Testing Differential Privacy

07/10/2023
by   William Kong, et al.
0

Governments and industries have widely adopted differential privacy as a measure to protect users' sensitive data, creating the need for new implementations of differentially private algorithms. In order to properly test and audit these algorithms, a suite of tools for testing the property of differential privacy is needed. In this work we expand this testing suite and introduce RényiTester, an algorithm that can verify if a mechanism is Rényi differentially private. Our algorithm computes computes a lower bound of the Rényi divergence between the distributions of a mechanism on neighboring datasets, only requiring black-box access to samples from the audited mechanism. We test this approach on a variety of pure and Rényi differentially private mechanisms with diverse output spaces and show that RényiTester detects bugs in mechanisms' implementations and design flaws. While detecting that a general mechanism is differentially private is known to be NP hard, we empirically show that tools like RényiTester provide a way for researchers and engineers to decrease the risk of deploying mechanisms that expose users' privacy.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset