Do Not Trust Prediction Scores for Membership Inference Attacks

11/17/2021
by   Dominik Hintersdorf, et al.
3

Membership inference attacks (MIAs) aim to determine whether a specific sample was used to train a predictive model. Knowing this may indeed lead to a privacy breach. Arguably, most MIAs, however, make use of the model's prediction scores - the probability of each output given some input - following the intuition that the trained model tends to behave differently on its training data. We argue that this is a fallacy for many modern deep network architectures, e.g., ReLU type neural networks produce almost always high prediction scores far away from the training data. Consequently, MIAs will miserably fail since this behavior leads to high false-positive rates not only on known domains but also on out-of-distribution data and implicitly acts as a defense against MIAs. Specifically, using generative adversarial networks, we are able to produce a potentially infinite number of samples falsely classified as part of the training data. In other words, the threat of MIAs is overestimated and less information is leaked than previously assumed. Moreover, there is actually a trade-off between the overconfidence of classifiers and their susceptibility to MIAs: the more classifiers know when they do not know, making low confidence predictions far away from the training data, the more they reveal the training data.

READ FULL TEXT

page 1

page 6

page 12

research
12/13/2018

Why ReLU networks yield high-confidence predictions far away from the training data and how to mitigate the problem

Classifiers used in the wild, in particular for safety-critical systems,...
research
05/29/2023

Membership Inference Attacks against Language Models via Neighbourhood Comparison

Membership Inference attacks (MIAs) aim to predict whether a data sample...
research
09/26/2019

Towards neural networks that provably know when they don't know

It has recently been shown that ReLU networks produce arbitrarily over-c...
research
12/06/2022

On the Discredibility of Membership Inference Attacks

With the wide-spread application of machine learning models, it has beco...
research
06/10/2022

Membership Inference via Backdooring

Recently issued data privacy regulations like GDPR (General Data Protect...
research
10/06/2020

Fixing Asymptotic Uncertainty of Bayesian Neural Networks with Infinite ReLU Features

Approximate Bayesian methods can mitigate overconfidence in ReLU network...
research
07/27/2022

Membership Inference Attacks via Adversarial Examples

The raise of machine learning and deep learning led to significant impro...

Please sign up or login with your details

Forgot password? Click here to reset