Bayesian Neural Networks: Essentials

06/22/2021
by   Daniel T Chang, et al.
0

Bayesian neural networks utilize probabilistic layers that capture uncertainty over weights and activations, and are trained using Bayesian inference. Since these probabilistic layers are designed to be drop-in replacement of their deterministic counter parts, Bayesian neural networks provide a direct and natural way to extend conventional deep neural networks to support probabilistic deep learning. However, it is nontrivial to understand, design and train Bayesian neural networks due to their complexities. We discuss the essentials of Bayesian neural networks including duality (deep neural networks, probabilistic models), approximate Bayesian inference, Bayesian priors, Bayesian posteriors, and deep variational learning. We use TensorFlow Probability APIs and code examples for illustration. The main problem with Bayesian neural networks is that the architecture of deep neural networks makes it quite redundant, and costly, to account for uncertainty for a large number of successive layers. Hybrid Bayesian neural networks, which use few probabilistic layers judicially positioned in the networks, provide a practical solution.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/31/2021

Probabilistic Deep Learning with Probabilistic Neural Networks and Deep Probabilistic Models

Probabilistic deep learning is deep learning that accounts for uncertain...
research
07/14/2021

Hybrid Bayesian Neural Networks with Functional Probabilistic Layers

Bayesian neural networks provide a direct and natural way to extend stan...
research
09/18/2017

ZhuSuan: A Library for Bayesian Deep Learning

In this paper we introduce ZhuSuan, a python probabilistic programming l...
research
11/22/2019

DBSN: Measuring Uncertainty through Bayesian Learning of Deep Neural Network Structures

Bayesian neural networks (BNNs) introduce uncertainty estimation to deep...
research
05/24/2020

Functional Space Variational Inference for Uncertainty Estimation in Computer Aided Diagnosis

Deep neural networks have revolutionized medical image analysis and dise...
research
08/26/2020

Uncertainty-Aware Surrogate Model For Oilfield Reservoir Simulation

Deep neural networks have gained increased attention in machine learning...
research
07/14/2020

Hands-on Bayesian Neural Networks – a Tutorial for Deep Learning Users

Modern deep learning methods have equipped researchers and engineers wit...

Please sign up or login with your details

Forgot password? Click here to reset