DeepAI AI Chat
Log In Sign Up

Uncertainty estimation for classification and risk prediction in medical settings

by   Lotta Meijerink, et al.

In a data-scarce field such as healthcare, where models often deliver predictions on patients with rare conditions, the ability to measure the uncertainty of a model's prediction could potentially lead to improved effectiveness of decision support tools and increased user trust. This work advances the understanding of uncertainty estimation for classification and risk prediction on medical tabular data, in a three-fold way. First, we analyze two families of promising methods and discuss the preferred approach for uncertainty estimation for classification and risk prediction. Second, these remarks are enriched by considerations of the interplay of uncertainty estimation with class imbalance, post-modeling calibration and other modeling procedures. Finally, we expand and refine the set of heuristics to select an uncertainty estimation technique, introducing tests for clinically-relevant scenarios such as generalization to uncommon pathologies, changes in clinical protocol and simulations of corrupted data. These findings are supported by an array of experiments on toy and real-world data


page 6

page 7

page 10


Trust Issues: Uncertainty Estimation Does Not Enable Reliable OOD Detection On Medical Tabular Data

When deploying machine learning models in high-stakes real-world environ...

Exploring Prediction Uncertainty in Machine Translation Quality Estimation

Machine Translation Quality Estimation is a notoriously difficult task, ...

Bayesian Modelling in Practice: Using Uncertainty to Improve Trustworthiness in Medical Applications

The Intensive Care Unit (ICU) is a hospital department where machine lea...

Leveraging Clinical Context for User-Centered Explainability: A Diabetes Use Case

Academic advances of AI models in high-precision domains, like healthcar...

Variational Disentanglement for Rare Event Modeling

Combining the increasing availability and abundance of healthcare data a...