DeepAI AI Chat
Log In Sign Up

Variational Bayes In Private Settings (VIPS)

11/01/2016
by   Mijung Park, et al.
University of Amsterdam
University of Maryland, Baltimore County
University of California, San Diego
0

We provide a general framework for privacy-preserving variational Bayes (VB) for a large class of probabilistic models, called the conjugate exponential (CE) family. Our primary observation is that when models are in the CE family, we can privatise the variational posterior distributions simply by perturbing the expected sufficient statistics of the complete-data likelihood. For widely used non-CE models with binomial likelihoods, we exploit the Pólya-Gamma data augmentation scheme to bring such models into the CE family, such that inferences in the modified model resemble the private variational Bayes algorithm as closely as possible. The iterative nature of variational Bayes presents a further challenge since iterations increase the amount of noise needed. We overcome this by combining: (1) a relaxed notion of differential privacy, called concentrated differential privacy, which provides a tight bound on the privacy cost of multiple VB iterations and thus significantly decreases the amount of additive noise; and (2) the privacy amplification effect of subsampling mini-batches from large-scale data in stochastic learning. We empirically demonstrate the effectiveness of our method in CE and non-CE models including latent Dirichlet allocation, Bayesian logistic regression, and sigmoid belief networks, evaluated on real-world datasets.

READ FULL TEXT
09/14/2016

Private Topic Modeling

We develop a privatised stochastic variational inference method for Late...
11/30/2017

Differentially Private Dropout

Large data collections required for the training of neural networks ofte...
11/30/2017

Differentially Private Variational Dropout

Deep neural networks with their large number of parameters are highly fl...
11/24/2015

Private Posterior distributions from Variational approximations

Privacy preserving mechanisms such as differential privacy inject additi...
02/25/2022

Does Label Differential Privacy Prevent Label Inference Attacks?

Label differential privacy (LDP) is a popular framework for training pri...
10/02/2017

Rényi Differential Privacy Mechanisms for Posterior Sampling

Using a recently proposed privacy definition of Rényi Differential Priva...
11/19/2017

A note on quadratic approximations of logistic log-likelihoods

Quadratic approximations of logistic log-likelihoods are fundamental to ...

Code Repositories