BNNpriors: A library for Bayesian neural network inference with different prior distributions

05/14/2021
by   Vincent Fortuin, et al.
0

Bayesian neural networks have shown great promise in many applications where calibrated uncertainty estimates are crucial and can often also lead to a higher predictive performance. However, it remains challenging to choose a good prior distribution over their weights. While isotropic Gaussian priors are often chosen in practice due to their simplicity, they do not reflect our true prior beliefs well and can lead to suboptimal performance. Our new library, BNNpriors, enables state-of-the-art Markov Chain Monte Carlo inference on Bayesian neural networks with a wide range of predefined priors, including heavy-tailed ones, hierarchical ones, and mixture priors. Moreover, it follows a modular approach that eases the design and implementation of new custom priors. It has facilitated foundational discoveries on the nature of the cold posterior effect in Bayesian neural networks and will hopefully catalyze future research as well as practical applications in this area.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
02/12/2021

Bayesian Neural Network Priors Revisited

Isotropic Gaussian priors are the de facto standard for modern Bayesian ...
research
11/25/2020

All You Need is a Good Functional Prior for Bayesian Deep Learning

The Bayesian treatment of neural networks dictates that a prior distribu...
research
06/13/2018

Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors

Bayesian Neural Networks (BNNs) have recently received increasing attent...
research
07/08/2020

Double spike Dirichlet priors for structured weighting

Assigning weights to a large pool of objects is a fundamental task in a ...
research
05/11/2020

Prior choice affects ability of Bayesian neural networks to identify unknowns

Deep Bayesian neural networks (BNNs) are a powerful tool, though computa...
research
10/12/2021

Uncertainty-based out-of-distribution detection requires suitable function space priors

The need to avoid confident predictions on unfamiliar data has sparked i...
research
12/20/2021

Bayesian neural network priors for edge-preserving inversion

We consider Bayesian inverse problems wherein the unknown state is assum...

Please sign up or login with your details

Forgot password? Click here to reset