Knowledge Removal in Sampling-based Bayesian Inference

03/24/2022
by   Shaopeng Fu, et al.
0

The right to be forgotten has been legislated in many countries, but its enforcement in the AI industry would cause unbearable costs. When single data deletion requests come, companies may need to delete the whole models learned with massive resources. Existing works propose methods to remove knowledge learned from data for explicitly parameterized models, which however are not appliable to the sampling-based Bayesian inference, i.e., Markov chain Monte Carlo (MCMC), as MCMC can only infer implicit distributions. In this paper, we propose the first machine unlearning algorithm for MCMC. We first convert the MCMC unlearning problem into an explicit optimization problem. Based on this problem conversion, an MCMC influence function is designed to provably characterize the learned knowledge from data, which then delivers the MCMC unlearning algorithm. Theoretical analysis shows that MCMC unlearning would not compromise the generalizability of the MCMC models. Experiments on Gaussian mixture models and Bayesian neural networks confirm the effectiveness of the proposed algorithm. The code is available at <https://github.com/fshp971/mcmc-unlearning>.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
01/16/2021

Bayesian Inference Forgetting

The right to be forgotten has been legislated in many countries but the ...
research
01/18/2022

Online, Informative MCMC Thinning with Kernelized Stein Discrepancy

A fundamental challenge in Bayesian inference is efficient representatio...
research
04/02/2023

Bayesian neural networks via MCMC: a Python-based tutorial

Bayesian inference provides a methodology for parameter estimation and u...
research
12/22/2021

A Comparison of Bayesian Inference Techniques for Sparse Factor Analysis

Dimension reduction algorithms aim to discover latent variables which de...
research
04/09/2023

PriorCVAE: scalable MCMC parameter inference with Bayesian deep generative modelling

In applied fields where the speed of inference and model flexibility are...
research
01/31/2023

Kernel Stein Discrepancy thinning: a theoretical perspective of pathologies and a practical fix with regularization

Stein thinning is a promising algorithm proposed by (Riabiz et al., 2022...
research
11/30/2018

Stochastic Gradient MCMC with Repulsive Forces

We propose a unifying view of two different families of Bayesian inferen...

Please sign up or login with your details

Forgot password? Click here to reset