DP-Fast MH: Private, Fast, and Accurate Metropolis-Hastings for Large-Scale Bayesian Inference

03/10/2023
by   Wanrong Zhang, et al.
0

Bayesian inference provides a principled framework for learning from complex data and reasoning under uncertainty. It has been widely applied in machine learning tasks such as medical diagnosis, drug design, and policymaking. In these common applications, the data can be highly sensitive. Differential privacy (DP) offers data analysis tools with powerful worst-case privacy guarantees and has been developed as the leading approach in privacy-preserving data analysis. In this paper, we study Metropolis-Hastings (MH), one of the most fundamental MCMC methods, for large-scale Bayesian inference under differential privacy. While most existing private MCMC algorithms sacrifice accuracy and efficiency to obtain privacy, we provide the first exact and fast DP MH algorithm, using only a minibatch of data in most iterations. We further reveal, for the first time, a three-way trade-off among privacy, scalability (i.e. the batch size), and efficiency (i.e. the convergence rate), theoretically characterizing how privacy affects the utility and computational cost in Bayesian inference. We empirically demonstrate the effectiveness and efficiency of our algorithm in various experiments.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
03/23/2016

On the Theory and Practice of Privacy-Preserving Bayesian Data Analysis

Bayesian inference has great promise for the privacy-preserving analysis...
research
06/17/2021

Differentially Private Hamiltonian Monte Carlo

Markov chain Monte Carlo (MCMC) algorithms have long been the main workh...
research
08/31/2023

Exact and Efficient Bayesian Inference for Privacy Risk Quantification (Extended Version)

Data analysis has high value both for commercial and research purposes. ...
research
04/06/2023

When approximate design for fast homomorphic computation provides differential privacy guarantees

While machine learning has become pervasive in as diversified fields as ...
research
03/22/2018

Locally Private Bayesian Inference for Count Models

As more aspects of social interaction are digitally recorded, there is a...
research
06/07/2021

Antipodes of Label Differential Privacy: PATE and ALIBI

We consider the privacy-preserving machine learning (ML) setting where t...
research
01/20/2023

Cohere: Privacy Management in Large Scale Systems

The need for a privacy management layer in today's systems started to ma...

Please sign up or login with your details

Forgot password? Click here to reset