Black-Box Audits for Group Distribution Shifts

09/08/2022
by   Marc Juarez, et al.
0

When a model informs decisions about people, distribution shifts can create undue disparities. However, it is hard for external entities to check for distribution shift, as the model and its training set are often proprietary. In this paper, we introduce and study a black-box auditing method to detect cases of distribution shift that lead to a performance disparity of the model across demographic groups. By extending techniques used in membership and property inference attacks – which are designed to expose private information from learned models – we demonstrate that an external auditor can gain the information needed to identify these distribution shifts solely by querying the model. Our experimental results on real-world datasets show that this approach is effective, achieving 80–100 underrepresentation of a demographic group in the training set. Researchers and investigative journalists can use our tools to perform non-collaborative audits of proprietary models and expose cases of underrepresentation in the training datasets.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/29/2019

White-box vs Black-box: Bayes Optimal Strategies for Membership Inference

Membership inference determines, given a sample and trained parameters o...
research
07/07/2023

Scalable Membership Inference Attacks via Quantile Regression

Membership inference attacks are designed to determine, using black box ...
research
02/04/2022

Discovering Distribution Shifts using Latent Space Representations

Rapid progress in representation learning has led to a proliferation of ...
research
06/01/2021

Adaptive Conformal Inference Under Distribution Shift

We develop methods for forming prediction sets in an online setting wher...
research
01/29/2019

Repairing without Retraining: Avoiding Disparate Impact with Counterfactual Distributions

When the average performance of a prediction model varies significantly ...
research
05/25/2023

Rectifying Group Irregularities in Explanations for Distribution Shift

It is well-known that real-world changes constituting distribution shift...
research
02/20/2023

Take Me Home: Reversing Distribution Shifts using Reinforcement Learning

Deep neural networks have repeatedly been shown to be non-robust to the ...

Please sign up or login with your details

Forgot password? Click here to reset