DeepAI AI Chat
Log In Sign Up

Learnable Uncertainty under Laplace Approximations

by   Agustinus Kristiadi, et al.

Laplace approximations are classic, computationally lightweight means for constructing Bayesian neural networks (BNNs). As in other approximate BNNs, one cannot necessarily expect the induced predictive uncertainty to be calibrated. Here we develop a formalism to explicitly "train" the uncertainty in a decoupled way to the prediction itself. To this end we introduce uncertainty units for Laplace-approximated networks: Hidden units with zero weights that can be added to any pre-trained, point-estimated network. Since these units are inactive, they do not affect the predictions. But their presence changes the geometry (in particular the Hessian) of the loss landscape around the point estimate, thereby affecting the network's uncertainty estimates under a Laplace approximation. We show that such units can be trained via an uncertainty-aware objective, making the Laplace approximation competitive with more expensive alternative uncertainty-quantification frameworks.


page 1

page 2

page 3

page 4


'In-Between' Uncertainty in Bayesian Neural Networks

We describe a limitation in the expressiveness of the predictive uncerta...

L2M: Practical posterior Laplace approximation with optimization-driven second moment estimation

Uncertainty quantification for deep neural networks has recently evolved...

Mixtures of Laplace Approximations for Improved Post-Hoc Uncertainty in Deep Learning

Deep neural networks are prone to overconfident predictions on outliers....

Bayesian Metric Learning for Uncertainty Quantification in Image Retrieval

We propose the first Bayesian encoder for metric learning. Rather than r...

Laplace Redux – Effortless Bayesian Deep Learning

Bayesian formulations of deep learning have been shown to have compellin...

Variational Linearized Laplace Approximation for Bayesian Deep Learning

Pre-trained deep neural networks can be adapted to perform uncertainty e...

Online Structured Laplace Approximations For Overcoming Catastrophic Forgetting

We introduce the Kronecker factored online Laplace approximation for ove...