DeepAI AI Chat
Log In Sign Up

Calibrating Ensembles for Scalable Uncertainty Quantification in Deep Learning-based Medical Segmentation

09/20/2022
by   Thomas Buddenkotte, et al.
0

Uncertainty quantification in automated image analysis is highly desired in many applications. Typically, machine learning models in classification or segmentation are only developed to provide binary answers; however, quantifying the uncertainty of the models can play a critical role for example in active learning or machine human interaction. Uncertainty quantification is especially difficult when using deep learning-based models, which are the state-of-the-art in many imaging applications. The current uncertainty quantification approaches do not scale well in high-dimensional real-world problems. Scalable solutions often rely on classical techniques, such as dropout, during inference or training ensembles of identical models with different random seeds to obtain a posterior distribution. In this paper, we show that these approaches fail to approximate the classification probability. On the contrary, we propose a scalable and intuitive framework to calibrate ensembles of deep learning models to produce uncertainty quantification measurements that approximate the classification probability. On unseen test data, we demonstrate improved calibration, sensitivity (in two out of three cases) and precision when being compared with the standard approaches. We further motivate the usage of our method in active learning, creating pseudo-labels to learn from unlabeled images and human-machine collaboration.

READ FULL TEXT
10/06/2020

Empirical Frequentist Coverage of Deep Learning Uncertainty Quantification Procedures

Uncertainty quantification for complex deep learning models is increasin...
09/12/2022

TEDL: A Two-stage Evidential Deep Learning Method for Classification Uncertainty Quantification

In this paper, we propose TEDL, a two-stage learning approach to quantif...
05/14/2020

Efficient and Scalable Bayesian Neural Nets with Rank-1 Factors

Bayesian neural networks (BNNs) demonstrate promising success in improvi...
09/13/2019

Evaluating and Boosting Uncertainty Quantification in Classification

Emergence of artificial intelligence techniques in biomedical applicatio...
04/22/2020

Deeply Uncertain: Comparing Methods of Uncertainty Quantification in Deep Learning Algorithms

We present a comparison of methods for uncertainty quantification (UQ) i...
05/28/2019

Generative Parameter Sampler For Scalable Uncertainty Quantification

Uncertainty quantification has been a core of the statistical machine le...