Differentially Private Learning Does Not Bound Membership Inference

10/23/2020
by   Thomas Humphries, et al.
0

Training machine learning models on privacy-sensitive data has become a popular practice, driving innovation in ever-expanding fields. This has opened the door to a series of new attacks, such as Membership Inference Attacks (MIAs), that exploit vulnerabilities in ML models in order to expose the privacy of individual training samples. A growing body of literature holds up Differential Privacy (DP) as an effective defense against such attacks, and companies like Google and Amazon include this privacy notion in their machine-learning-as-a-service products. However, little scrutiny has been given to how underlying correlations within the datasets used for training these models can impact the privacy guarantees provided by DP. In this work, we challenge prior findings that suggest DP provides a strong defense against MIAs. We provide theoretical and experimental evidence for cases where the theoretical bounds of DP are violated by MIAs using the same attacks described in prior work. We show this hypothetically with artificial, pathological datasets as well as with real-world datasets carefully split to create a distinction between member and non-member samples. Our findings suggest that certain properties of datasets, such as bias or data correlation, play a critical role in determining the effectiveness of DP as a privacy preserving mechanism against MIAs. Further, ensuring that a given dataset is resilient against these MIAs may be virtually impossible for a data analyst to determine.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/24/2022

Bounding Membership Inference

Differential Privacy (DP) is the de facto standard for reasoning about t...
research
12/06/2018

Differentially Private Data Generative Models

Deep neural networks (DNNs) have recently been widely adopted in various...
research
01/24/2020

Privacy for All: Demystify Vulnerability Disparity of Differential Privacy against Membership Inference Attack

Machine learning algorithms, when applied to sensitive data, pose a pote...
research
11/21/2022

Privacy in Practice: Private COVID-19 Detection in X-Ray Images

Machine learning (ML) can help fight the COVID-19 pandemic by enabling r...
research
11/12/2022

Provable Membership Inference Privacy

In applications involving sensitive data, such as finance and healthcare...
research
07/06/2021

DTGAN: Differential Private Training for Tabular GANs

Tabular generative adversarial networks (TGAN) have recently emerged to ...
research
06/05/2023

Discriminative Adversarial Privacy: Balancing Accuracy and Membership Privacy in Neural Networks

The remarkable proliferation of deep learning across various industries ...

Please sign up or login with your details

Forgot password? Click here to reset