Objective Evaluation of Deep Uncertainty Predictions for COVID-19 Detection

12/22/2020
by   Hamzeh Asgharnezhad, et al.
13

Deep neural networks (DNNs) have been widely applied for detecting COVID-19 in medical images. Existing studies mainly apply transfer learning and other data representation strategies to generate accurate point estimates. The generalization power of these networks is always questionable due to being developed using small datasets and failing to report their predictive confidence. Quantifying uncertainties associated with DNN predictions is a prerequisite for their trusted deployment in medical settings. Here we apply and evaluate three uncertainty quantification techniques for COVID-19 detection using chest X-Ray (CXR) images. The novel concept of uncertainty confusion matrix is proposed and new performance metrics for the objective evaluation of uncertainty estimates are introduced. Through comprehensive experiments, it is shown that networks pertained on CXR images outperform networks pretrained on natural image datasets such as ImageNet. Qualitatively and quantitatively evaluations also reveal that the predictive uncertainty estimates are statistically higher for erroneous predictions than correct predictions. Accordingly, uncertainty quantification methods are capable of flagging risky predictions with high uncertainty estimates. We also observe that ensemble methods more reliably capture uncertainties during the inference.

READ FULL TEXT

page 1

page 3

page 4

research
07/28/2021

Uncertainty-Aware Credit Card Fraud Detection Using Deep Learning

Countless research works of deep neural networks (DNNs) in the task of c...
research
07/19/2021

Confidence Aware Neural Networks for Skin Cancer Detection

Deep learning (DL) models have received particular attention in medical ...
research
11/17/2022

Fast Uncertainty Estimates in Deep Learning Interatomic Potentials

Deep learning has emerged as a promising paradigm to give access to high...
research
07/07/2023

URL: A Representation Learning Benchmark for Transferable Uncertainty Estimates

Representation learning has significantly driven the field to develop pr...
research
06/15/2020

Can Your AI Differentiate Cats from Covid-19? Sample Efficient Uncertainty Estimation for Deep Learning Safety

Deep Neural Networks (DNNs) are known to make highly overconfident predi...
research
07/24/2021

An Uncertainty-Aware Deep Learning Framework for Defect Detection in Casting Products

Defects are unavoidable in casting production owing to the complexity of...
research
02/02/2023

Generalized Uncertainty of Deep Neural Networks: Taxonomy and Applications

Deep neural networks have seen enormous success in various real-world ap...

Please sign up or login with your details

Forgot password? Click here to reset