Discovering Inductive Bias with Gibbs Priors: A Diagnostic Tool for Approximate Bayesian Inference

03/07/2022
by   Luca Rendsburg, et al.
0

Full Bayesian posteriors are rarely analytically tractable, which is why real-world Bayesian inference heavily relies on approximate techniques. Approximations generally differ from the true posterior and require diagnostic tools to assess whether the inference can still be trusted. We investigate a new approach to diagnosing approximate inference: the approximation mismatch is attributed to a change in the inductive bias by treating the approximations as exact and reverse-engineering the corresponding prior. We show that the problem is more complicated than it appears to be at first glance, because the solution generally depends on the observation. By reframing the problem in terms of incompatible conditional distributions we arrive at a natural solution: the Gibbs prior. The resulting diagnostic is based on pseudo-Gibbs sampling, which is widely applicable and easy to implement. We illustrate how the Gibbs prior can be used to discover the inductive bias in a controlled Gaussian setting and for a variety of Bayesian models and approximations.

READ FULL TEXT
research
06/19/2020

Distortion estimates for approximate Bayesian inference

Current literature on posterior approximation for Bayesian inference off...
research
08/10/2021

Robust posterior inference for Youden's index cutoff

Youden's index cutoff is a classifier mapping a patient's diagnostic tes...
research
11/24/2018

Amortized Bayesian inference for clustering models

We develop methods for efficient amortized approximate Bayesian inferenc...
research
07/02/2019

Adaptive particle-based approximations of the Gibbs posterior for inverse problems

In this work, we adopt a general framework based on the Gibbs posterior ...
research
09/14/2021

Gibbs posterior inference on a Levy density under discrete sampling

In mathematical finance, Levy processes are widely used for their abilit...
research
03/05/2021

On the Occasional Exactness of the Distributional Transform Approximation for Direct Gaussian Copula Models with Discrete Margins

The direct Gaussian copula model with discrete marginal distributions is...
research
11/20/2019

Assessment and adjustment of approximate inference algorithms using the law of total variance

A common method for assessing validity of Bayesian sampling or approxima...

Please sign up or login with your details

Forgot password? Click here to reset