Posterior Refinement Improves Sample Efficiency in Bayesian Neural Networks

05/20/2022
by   Agustinus Kristiadi, et al.
0

Monte Carlo (MC) integration is the de facto method for approximating the predictive distribution of Bayesian neural networks (BNNs). But, even with many MC samples, Gaussian-based BNNs could still yield bad predictive performance due to the posterior approximation's error. Meanwhile, alternatives to MC integration tend to be more expensive or biased. In this work, we experimentally show that the key to good MC-approximated predictive distributions is the quality of the approximate posterior itself. However, previous methods for obtaining accurate posterior approximations are expensive and non-trivial to implement. We, therefore, propose to refine Gaussian approximate posteriors with normalizing flows. When applied to last-layer BNNs, it yields a simple post hoc method for improving pre-existing parametric approximations. We show that the resulting posterior approximation is competitive with even the gold-standard full-batch Hamiltonian Monte Carlo.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/29/2019

Accelerating Monte Carlo Bayesian Inference via Approximating Predictive Uncertainty over Simplex

Estimating the uncertainty of a Bayesian model has been investigated for...
research
06/16/2023

Collapsed Inference for Bayesian Deep Learning

Bayesian neural networks (BNNs) provide a formalism to quantify and cali...
research
02/19/2022

Graph Reparameterizations for Enabling 1000+ Monte Carlo Iterations in Bayesian Deep Neural Networks

Uncertainty estimation in deep models is essential in many real-world ap...
research
04/08/2019

Bayesian Neural Networks at Finite Temperature

We recapitulate the Bayesian formulation of neural network based classif...
research
03/29/2021

Rapid Risk Minimization with Bayesian Models Through Deep Learning Approximation

In this paper, we introduce a novel combination of Bayesian Models (BMs)...
research
08/02/2019

Why Simple Quadrature is just as good as Monte Carlo

We motive and calculate Newton-Cotes quadrature integration variance and...
research
10/14/2021

Divide-and-Conquer Monte Carlo Fusion

Combining several (sample approximations of) distributions, which we ter...

Please sign up or login with your details

Forgot password? Click here to reset