Visualizing Uncertainty and Saliency Maps of Deep Convolutional Neural Networks for Medical Imaging Applications

07/05/2019 ∙ by Jae Duk Seo, et al. ∙ 0

Deep learning models are now used in many different industries, while in certain domains safety is not a critical issue in the medical field it is a huge concern. Not only, we want the models to generalize well but we also want to know the models confidence respect to its decision and which features matter the most. Our team aims to develop a full pipeline in which not only displays the uncertainty of the models decision but also, the saliency map to show which sets of pixels of the input image contribute most to the predictions.



There are no comments yet.


page 1

page 2

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

The use of machine learning models in bio-medical imaging has ever been growing since the break through of Image-net challenge in 2014. However, interpret-ability of these models are still a huge problem for industry adoption. Recently developed gradient method such as (Simonyan et al., 2013; Springenberg et al., 2014; Sundararajan et al., 2017), reveals which part of the image the network has focused on to make either segmentation or classification decision. While (Gal & Ghahramani, 2016)

have created a theoretical framework for justifying the use of drop-out as uncertainty estimation.

Figure 1: Segmentation result for Retinal Images along with the models Epistemic and Aleatoric uncertainty

2 Problem Description

Statistical models, such as deep neural networks, that are used in medical imaging domain must be transparent and interpretable. One way of achieving this is by outputting the model’s confidence with it’s decision as well as visualize which part of the original image did the model take into account the most. To best of our knowledge, this is the first attempt to predict model’s uncertainty while producing it’s saliency map. This is a problem since, the current standard for measuring performance of these models is classification or segmentation accuracy. While those measurement’s are important, it is not the full picture. To insure trust from the patient and the medical professional, expert systems need to report their inner working mechanisms.

3 Design Solution

Thanks to recent theoretical development done related to Bayesian neural networks now it is possible to produce both epistemic and aleatoric uncertainty. Additionally, methods that reveals which input features contributed the most for network’s decision have been developed, as early as back in 2014. We aim to combine both of these procedures as a small step towards creating transparent machine learning models for medical imaging.

3.1 Bayesian Deep Learning

There are two different types of uncertainty, epistemic uncertainty which captures the confidence within the used statistical model. This uncertainty can be explained away given enough data, and is often referred to as model uncertainty. And aleatoric uncertainty captures the uncertainty with respect to information which our data cannot explain. And based on the theoretical development of a methodology called ’Dropout as a Bayesian Approximation’ both can be measured using drop-out.

3.2 Saliency Mapping

For both classification and segmentation results knowing which part of the input data is mostly important to the model is extremely beneficial. Especially for medical diagnosis, the medical professional can actually see if the model is focusing on the right places. Here we apply the gradient methods such as integral gradients, to visualize which part of the input matters the most.

Figure 2: Different Saliency mapping visualized for chest X-Ray classification

4 Application

4.1 Segmentation of Blood Vessels in Retinal Images

Using the data from (Staal et al., 2004) we trained a nine layer Unet(Ronneberger et al., 2015) to perform segmentation. Which the objective function was set as the Dice coefficient loss. The model was able to achieve 0.22 loss on training set and 0.26 on validation data, indicating a high generalization power of the model. And as depicted in figure 1 by using the Monte Carlo Dropout Sampling method we can output the model’s uncertainty. We can notice when the model is uncertain about the region of the blood vessel, it tends to produce very thin segmentation maps.

4.2 Classification on Chest X-ray images

Next we used a pre-trained VGG 16 network(Simonyan & Zisserman, 2015), and only trained a top layer to perform classification on Chest X-ray (Wang et al., 2017) data set. And as seen in figure 2, we can observe the location where the model focuses on for classification. And each of the saliency mapping methods (and it’s smoother version) gives consistent results, indicating that the left bottom region of the image is a critical region.

5 Implementation

Due to restrictions related to medical imaging data, it is not realistic or feasible to expect multiple health institution to share their data. However, thanks to increase in the demand of data scientist in many health organization it is reasonable to expect that each of those organization would be interested in applying a deep learning models as possible solutions. Hence we see the most optimal approach is to create a step by step tutorial series, either in a video format or a single web-page format, so that different organizations can implement this solution in their current system. Additionally, we are aiming to expand the example use-case of these methods, specifically, we are planning to use neural networks to achieve dual task on microscopic cell images. Not only we can perform segmentation of cell regions but also count the number of cells in a particular image, while outputting uncertainty for each measurements. Finally, possible future works includes, creating a proof of concept web application where medical professionals can upload their own data and see the promising result of this methodology.


This research was supported by Ryerson Vision lab, and the author of this paper wishes to express appreciations toward Dr. Neil Bruce and Dr. Kosta Derpanis for providing a comfortable lab space.