'In-Between' Uncertainty in Bayesian Neural Networks

06/27/2019
by   Andrew Y. K. Foong, et al.
1

We describe a limitation in the expressiveness of the predictive uncertainty estimate given by mean-field variational inference (MFVI), a popular approximate inference method for Bayesian neural networks. In particular, MFVI fails to give calibrated uncertainty estimates in between separated regions of observations. This can lead to catastrophically overconfident predictions when testing on out-of-distribution data. Avoiding such overconfidence is critical for active learning, Bayesian optimisation and out-of-distribution robustness. We instead find that a classical technique, the linearised Laplace approximation, can handle 'in-between' uncertainty much better for small network architectures.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/02/2019

Pathologies of Factorised Gaussian and MC Dropout Posteriors in Bayesian Neural Networks

Neural networks provide state-of-the-art performance on a variety of tas...
research
10/06/2020

Learnable Uncertainty under Laplace Approximations

Laplace approximations are classic, computationally lightweight means fo...
research
02/25/2020

Variational Inference and Bayesian CNNs for Uncertainty Estimation in Multi-Factorial Bone Age Prediction

Additionally to the extensive use in clinical medicine, biological age (...
research
05/11/2017

Bayesian Approaches to Distribution Regression

Distribution regression has recently attracted much interest as a generi...
research
06/02/2022

BayesFormer: Transformer with Uncertainty Estimation

Transformer has become ubiquitous due to its dominant performance in var...
research
11/06/2020

Beyond Marginal Uncertainty: How Accurately can Bayesian Regression Models Estimate Posterior Predictive Correlations?

While uncertainty estimation is a well-studied topic in deep learning, m...
research
02/27/2021

Variational Laplace for Bayesian neural networks

We develop variational Laplace for Bayesian neural networks (BNNs) which...

Please sign up or login with your details

Forgot password? Click here to reset