Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex

05/29/2019
by   Yufei Cui, et al.
0

Estimating the uncertainty of a Bayesian model has been investigated for decades. The model posterior is almost always intractable, such that approximation is necessary. In many real-world cases, even though a decent estimation of the model posterior is obtained, another approximation is required to compute the predictive distribution over the desired output. A common accurate solution is to use Monte Carlo (MC) integration. However, it needs to maintain a large number of samples, evaluate the model repeatedly and average multiple model outputs. In this paper, we propose a method to approximate the probability distribution over the simplex induced by model posterior, enabling tractable computation of the predictive distribution for classification. The aim is to approximate the induced uncertainty of a specific Bayesian model, meanwhile alleviating the heavy workload of MC integration in testing time. Methodologically, we adapt Wasserstein distance to learn the induced conditional distributions, which is novel for Bayesian learning. The proposed method is universally applicable to Bayesian classification models that allow for posterior sampling. Empirical results validate the strong practical performance of our approach.

READ FULL TEXT
research
05/20/2022

Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks

Monte Carlo (MC) integration is the de facto method for approximating th...
research
11/26/2018

Non-deterministic inference using random set models: theory, approximation, and sampling method

A random set is a generalisation of a random variable, i.e. a set-valued...
research
02/07/2022

Theoretical characterization of uncertainty in high-dimensional linear classification

Being able to reliably assess not only the accuracy but also the uncerta...
research
03/29/2021

Rapid Risk Minimization with Bayesian Models Through Deep Learning Approximation

In this paper, we introduce a novel combination of Bayesian Models (BMs)...
research
07/18/2021

Compressed particle methods for expensive models with application in Astronomy and Remote Sensing

In many inference problems, the evaluation of complex and costly models ...
research
06/04/2019

Assessing the Robustness of Bayesian Dark Knowledge to Posterior Uncertainty

Bayesian Dark Knowledge is a method for compressing the posterior predic...
research
04/08/2019

Bayesian Neural Networks at Finite Temperature

We recapitulate the Bayesian formulation of neural network based classif...

Please sign up or login with your details

Forgot password? Click here to reset