Correlated Parameters to Accurately Measure Uncertainty in Deep Neural Networks

04/02/2019
by   Konstantin Posch, et al.
0

In this article a novel approach for training deep neural networks using Bayesian techniques is presented. The Bayesian methodology allows for an easy evaluation of model uncertainty and additionally is robust to overfitting. These are commonly the two main problems classical, i.e. non-Bayesian, architectures have to struggle with. The proposed approach applies variational inference in order to approximate the intractable posterior distribution. In particular, the variational distribution is defined as product of multiple multivariate normal distributions with tridiagonal covariance matrices. Each single normal distribution belongs either to the weights, or to the biases corresponding to one network layer. The layer-wise a posteriori variances are defined based on the corresponding expectation values and further the correlations are assumed to be identical. Therefore, only a few additional parameters need to be optimized compared to non-Bayesian settings. The novel approach is successfully evaluated on basis of the popular benchmark datasets MNIST and CIFAR-10.

READ FULL TEXT

page 3

page 5

page 6

page 7

page 8

page 9

page 10

page 13

research
02/26/2019

Variational Inference to Measure Model Uncertainty in Deep Neural Networks

We present a novel approach for training deep neural networks in a Bayes...
research
10/26/2020

Scalable Bayesian neural networks by layer-wise input augmentation

We introduce implicit Bayesian neural networks, a simple and scalable ap...
research
09/13/2019

Adversarial α-divergence Minimization for Bayesian Approximate Inference

Neural networks are popular models for regression. They are often traine...
research
03/15/2016

Structured and Efficient Variational Deep Learning with Matrix Gaussian Posteriors

We introduce a variational Bayesian neural network where the parameters ...
research
10/24/2020

Implicit Variational Inference: the Parameter and the Predictor Space

Having access to accurate confidence levels along with the predictions a...
research
06/06/2015

Bayesian Convolutional Neural Networks with Bernoulli Approximate Variational Inference

Convolutional neural networks (CNNs) work well on large datasets. But la...
research
07/20/2021

A Bayesian Approach to Invariant Deep Neural Networks

We propose a novel Bayesian neural network architecture that can learn i...

Please sign up or login with your details

Forgot password? Click here to reset