Privacy Aware Experimentation over Sensitive Groups: A General Chi Square Approach

by   Rina Friedberg, et al.

We study a new privacy model where users belong to certain sensitive groups and we would like to conduct statistical inference on whether there is significant differences in outcomes between the various groups. In particular we do not consider the outcome of users to be sensitive, rather only the membership to certain groups. This is in contrast to previous work that has considered locally private statistical tests, where outcomes and groups are jointly privatized, as well as private A/B testing where the groups are considered public (control and treatment groups) while the outcomes are privatized. We cover several different settings of hypothesis tests after group membership has been privatized amongst the samples, including binary and real valued outcomes. We adopt the generalized χ^2 testing framework used in other works on hypothesis testing in different privacy models, which allows us to cover Z-tests, χ^2 tests for independence, t-tests, and ANOVA tests with a single unified approach. When considering two groups, we derive confidence intervals for the true difference in means and show traditional approaches for computing confidence intervals miss the true difference when privacy is introduced. For more than two groups, we consider several mechanisms for privatizing the group membership, showing that we can improve statistical power over the traditional tests that ignore the noise due to privacy. We also consider the application to private A/B testing to determine whether there is a significant change in the difference in means across sensitive groups between the control and treatment.


page 1

page 2

page 3

page 4


Fast computation of exact confidence intervals for randomized experiments with binary outcomes

Given a randomized experiment with binary outcomes, exact confidence int...

Locally private nonparametric confidence intervals and sequences

This work derives methods for performing nonparametric, nonasymptotic st...

The Structure of Optimal Private Tests for Simple Hypotheses

Hypothesis testing plays a central role in statistical inference, and is...

Are You Sure You're Sure? – Effects of Visual Representation on the Cliff Effect in Statistical Inference

Common reporting styles of statistical results, such as confidence inter...

Bayesian Estimation of Differential Privacy

Algorithms such as Differentially Private SGD enable training machine le...

SIMPLE-RC: Group Network Inference with Non-Sharp Nulls and Weak Signals

Large-scale network inference with uncertainty quantification has import...

Auditing ML Models for Individual Bias and Unfairness

We consider the task of auditing ML models for individual bias/unfairnes...

Please sign up or login with your details

Forgot password? Click here to reset