Performance Measurement for Deep Bayesian Neural Network

03/20/2019
by   Yikuan Li, et al.
0

Deep Bayesian neural network has aroused a great attention in recent years since it combines the benefits of deep neural network and probability theory. Because of this, the network can make predictions and quantify the uncertainty of the predictions at the same time, which is important in many life-threatening areas. However, most of the recent researches are mainly focusing on making the Bayesian neural network easier to train, and proposing methods to estimate the uncertainty. I notice there are very few works that properly discuss the ways to measure the performance of the Bayesian neural network. Although accuracy and average uncertainty are commonly used for now, they are too general to provide any insight information about the model. In this paper, we would like to introduce more specific criteria and propose several metrics to measure the model performance from different perspectives, which include model calibration measurement, data rejection ability and uncertainty divergence for samples from the same and different distributions.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
04/01/2022

DBCal: Density Based Calibration of classifier predictions for uncertainty quantification

Measurement of uncertainty of predictions from machine learning methods ...
research
12/03/2019

On the Validity of Bayesian Neural Networks for Uncertainty Estimation

Deep neural networks (DNN) are versatile parametric models utilised succ...
research
10/29/2018

Principled Uncertainty Estimation for Deep Neural Networks

When the cost of misclassifying a sample is high, it is useful to have a...
research
09/28/2017

Distance-based Confidence Score for Neural Network Classifiers

The reliable measurement of confidence in classifiers' predictions is ve...
research
03/05/2019

Evaluation of Neural Network Uncertainty Estimation with Application to Resource-Constrained Platforms

The ability to accurately estimate uncertainties in neural network predi...
research
02/23/2022

Using Bayesian Deep Learning to infer Planet Mass from Gaps in Protoplanetary Disks

Planet induced sub-structures, like annular gaps, observed in dust emiss...
research
05/26/2021

Estimating the Uncertainty of Neural Network Forecasts for Influenza Prevalence Using Web Search Activity

Influenza is an infectious disease with the potential to become a pandem...

Please sign up or login with your details

Forgot password? Click here to reset