Objective Evaluation of Deep Uncertainty Predictions for COVID-19 Detection

by   Hamzeh Asgharnezhad, et al.

Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray (CXR) images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced. Through comprehensive experiments, it is shown that networks pertained on CXR images outperform networks pretrained on natural image datasets such as ImageNet. Qualitatively and quantitatively evaluations also reveal that the predictive uncertainty estimates are statistically higher for erroneous predictions than correct predictions. Accordingly, uncertainty quantification methods are capable of flagging risky predictions with high uncertainty estimates. We also observe that ensemble methods more reliably capture uncertainties during the inference.


page 1

page 3

page 4


Uncertainty-Aware Credit Card Fraud Detection Using Deep Learning

Countless research works of deep neural networks (DNNs) in the task of c...

Confidence Aware Neural Networks for Skin Cancer Detection

Deep learning (DL) models have received particular attention in medical ...

Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety

Deep Neural Networks (DNNs) are known to make highly overconfident predi...

An Uncertainty-Aware Deep Learning Framework for Defect Detection in Casting Products

Defects are unavoidable in casting production owing to the complexity of...

Evaluating and Boosting Uncertainty Quantification in Classification

Emergence of artificial intelligence techniques in biomedical applicatio...

Uncertainty-Informed Deep Learning Models Enable High-Confidence Predictions for Digital Histopathology

A model's ability to express its own predictive uncertainty is an essent...

Supervised Uncertainty Quantification for Segmentation with Multiple Annotations

The accurate estimation of predictive uncertainty carries importance in ...