Calibration of Deep Probabilistic Models with Decoupled Bayesian Neural Networks

08/23/2019
by   Juan Maroñas, et al.
0

Deep Neural Networks (DNNs) have achieved state-of-the-art accuracy performance in many tasks. However, recent works have pointed out that the outputs provided by these models are not well-calibrated, seriously limiting their use in critical decision scenarios. In this work, we propose to use a decoupled Bayesian stage, implemented with a Bayesian Neural Network (BNN), to map the uncalibrated probabilities provided by a DNN to calibrated ones, consistently improving calibration. Our results evidence that incorporating uncertainty provides more reliable probabilistic models, a critical condition for achieving good calibration. We report a generous collection of experimental results using high-accuracy DNNs in standardized image classification benchmarks, showing the good performance, flexibility and robust behavior of our approach with respect to several state-of-the-art calibration methods. Code for reproducibility is provided.

READ FULL TEXT

page 5

page 10

research
09/29/2022

Bayesian Neural Network Versus Ex-Post Calibration For Prediction Uncertainty

Probabilistic predictions from neural networks which account for predict...
research
05/15/2022

Evaluating Uncertainty Calibration for Open-Set Recognition

Despite achieving enormous success in predictive accuracy for visual cla...
research
03/22/2020

Improving Calibration in Mixup-trained Deep Neural Networks through Confidence-Based Loss Functions

Deep Neural Networks (DNN) represent the state of the art in many tasks....
research
06/15/2021

Revisiting the Calibration of Modern Neural Networks

Accurate estimation of predictive uncertainty (model calibration) is ess...
research
08/02/2023

Calibration in Deep Learning: A Survey of the State-of-the-Art

Calibrating deep neural models plays an important role in building relia...
research
02/21/2020

Calibrating Deep Neural Networks using Focal Loss

Miscalibration – a mismatch between a model's confidence and its correct...
research
01/13/2021

Should Ensemble Members Be Calibrated?

Underlying the use of statistical approaches for a wide range of applica...

Please sign up or login with your details

Forgot password? Click here to reset