A Bayesian Neural Network based on Dropout Regulation

02/03/2021
by   Claire Theobald, et al.
0

Bayesian Neural Networks (BNN) have recently emerged in the Deep Learning world for dealing with uncertainty estimation in classification tasks, and are used in many application domains such as astrophysics, autonomous driving...BNN assume a prior over the weights of a neural network instead of point estimates, enabling in this way the estimation of both aleatoric and epistemic uncertainty of the model prediction.Moreover, a particular type of BNN, namely MC Dropout, assumes a Bernoulli distribution on the weights by using Dropout.Several attempts to optimize the dropout rate exist, e.g. using a variational approach.In this paper, we present a new method called "Dropout Regulation" (DR), which consists of automatically adjusting the dropout rate during training using a controller as used in automation.DR allows for a precise estimation of the uncertainty which is comparable to the state-of-the-art while remaining simple to implement.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/29/2018

Efficient Uncertainty Estimation for Semantic Segmentation in Videos

Uncertainty estimation in deep learning becomes more important recently....
10/12/2021

Robust Neural Regression via Uncertainty Learning

Deep neural networks tend to underestimate uncertainty and produce overl...
03/06/2020

Dropout Strikes Back: Improved Uncertainty Estimation via Diversity Sampled Implicit Ensembles

Modern machine learning models usually do not extrapolate well, i.e., th...
05/20/2015

Weight Uncertainty in Neural Networks

We introduce a new, efficient, principled and backpropagation-compatible...
05/13/2018

Spatial Uncertainty Sampling for End-to-End Control

End-to-end trained neural networks (NNs) are a compelling approach to au...
09/16/2020

Geometric Uncertainty in Patient-Specific Cardiovascular Modeling with Convolutional Dropout Networks

We propose a novel approach to generate samples from the conditional dis...
11/19/2019

Parameters Estimation for the Cosmic Microwave Background with Bayesian Neural Networks

In this paper, we present the first study that compares different models...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.