Structured Variational Learning of Bayesian Neural Networks with Horseshoe Priors

06/13/2018
by   Soumya Ghosh, et al.
0

Bayesian Neural Networks (BNNs) have recently received increasing attention for their ability to provide well-calibrated posterior uncertainties. However, model selection---even choosing the number of nodes---remains an open question. Recent work has proposed the use of a horseshoe prior over node pre-activations of a Bayesian neural network, which effectively turns off nodes that do not help explain the data. In this work, we propose several modeling and inference advances that consistently improve the compactness of the model learned while maintaining predictive performance, especially in smaller-sample settings including reinforcement learning.

READ FULL TEXT

page 7

page 8

page 13

page 14

research
05/29/2017

Model Selection in Bayesian Neural Networks via Horseshoe Priors

Bayesian Neural Networks (BNNs) have recently received increasing attent...
research
05/14/2021

BNNpriors: A library for Bayesian neural network inference with different prior distributions

Bayesian neural networks have shown great promise in many applications w...
research
12/06/2019

Sampling-Free Learning of Bayesian Quantized Neural Networks

Bayesian learning of model parameters in neural networks is important in...
research
08/25/2021

Layer Adaptive Node Selection in Bayesian Neural Networks: Statistical Guarantees and Implementation Details

Sparse deep neural networks have proven to be efficient for predictive m...
research
05/23/2022

Split personalities in Bayesian Neural Networks: the case for full marginalisation

The true posterior distribution of a Bayesian neural network is massivel...
research
05/23/2023

An Improved Variational Approximate Posterior for the Deep Wishart Process

Deep kernel processes are a recently introduced class of deep Bayesian m...
research
02/07/2019

Radial and Directional Posteriors for Bayesian Neural Networks

We propose a new variational family for Bayesian neural networks. We dec...

Please sign up or login with your details

Forgot password? Click here to reset