MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy

In some misspecified settings, the posterior distribution in Bayesian statistics may lead to inconsistent estimates. To fix this issue, it has been suggested to replace the likelihood by a pseudo-likelihood, that is the exponential of a loss function enjoying suitable robustness properties. In this paper, we build a pseudo-likelihood based on the Maximum Mean Discrepancy, defined via an embedding of probability distributions into a reproducing kernel Hilbert space. We show that this MMD-Bayes posterior is consistent and robust to model misspecification. As the posterior obtained in this way might be intractable, we also prove that reasonable variational approximations of this posterior enjoy the same properties. We provide details on a stochastic gradient algorithm to compute these variational approximations. Numerical simulations indeed suggest that our estimator is more robust to misspecification than the ones based on the likelihood.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/03/2021

Variational Bayes algorithm and posterior consistency of Ising model parameter estimation

Ising models originated in statistical physics and are widely used in mo...
research
06/13/2019

Statistical Inference for Generative Models with Maximum Mean Discrepancy

While likelihood-based inference and its variants provide a statisticall...
research
07/12/2021

Constrained Optimal Smoothing and Bayesian Estimation

In this paper, we extend the correspondence between Bayesian estimation ...
research
10/24/2021

A deep learning based surrogate model for stochastic simulators

We propose a deep learning-based surrogate model for stochastic simulato...
research
01/13/2021

PAC-Bayes Bounds on Variational Tempered Posteriors for Markov Models

Datasets displaying temporal dependencies abound in science and engineer...
research
05/09/2017

Frequentist Consistency of Variational Bayes

A key challenge for modern Bayesian statistics is how to perform scalabl...
research
12/12/2019

Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence

Many works in statistics aim at designing a universal estimation procedu...

Please sign up or login with your details

Forgot password? Click here to reset