The use of machine learning models in bio-medical imaging has ever been growing since the break through of Image-net challenge in 2014. However, interpret-ability of these models are still a huge problem for industry adoption. Recently developed gradient method such as (Simonyan et al., 2013; Springenberg et al., 2014; Sundararajan et al., 2017), reveals which part of the image the network has focused on to make either segmentation or classification decision. While (Gal & Ghahramani, 2016)
have created a theoretical framework for justifying the use of drop-out as uncertainty estimation.
2 Problem Description
Statistical models, such as deep neural networks, that are used in medical imaging domain must be transparent and interpretable. One way of achieving this is by outputting the model’s confidence with it’s decision as well as visualize which part of the original image did the model take into account the most. To best of our knowledge, this is the first attempt to predict model’s uncertainty while producing it’s saliency map. This is a problem since, the current standard for measuring performance of these models is classification or segmentation accuracy. While those measurement’s are important, it is not the full picture. To insure trust from the patient and the medical professional, expert systems need to report their inner working mechanisms.
3 Design Solution
Thanks to recent theoretical development done related to Bayesian neural networks now it is possible to produce both epistemic and aleatoric uncertainty. Additionally, methods that reveals which input features contributed the most for network’s decision have been developed, as early as back in 2014. We aim to combine both of these procedures as a small step towards creating transparent machine learning models for medical imaging.
3.1 Bayesian Deep Learning
There are two different types of uncertainty, epistemic uncertainty which captures the confidence within the used statistical model. This uncertainty can be explained away given enough data, and is often referred to as model uncertainty. And aleatoric uncertainty captures the uncertainty with respect to information which our data cannot explain. And based on the theoretical development of a methodology called ’Dropout as a Bayesian Approximation’ both can be measured using drop-out.
3.2 Saliency Mapping
For both classification and segmentation results knowing which part of the input data is mostly important to the model is extremely beneficial. Especially for medical diagnosis, the medical professional can actually see if the model is focusing on the right places. Here we apply the gradient methods such as integral gradients, to visualize which part of the input matters the most.
4.1 Segmentation of Blood Vessels in Retinal Images
Using the data from (Staal et al., 2004) we trained a nine layer Unet(Ronneberger et al., 2015) to perform segmentation. Which the objective function was set as the Dice coefficient loss. The model was able to achieve 0.22 loss on training set and 0.26 on validation data, indicating a high generalization power of the model. And as depicted in figure 1 by using the Monte Carlo Dropout Sampling method we can output the model’s uncertainty. We can notice when the model is uncertain about the region of the blood vessel, it tends to produce very thin segmentation maps.
4.2 Classification on Chest X-ray images
Next we used a pre-trained VGG 16 network(Simonyan & Zisserman, 2015), and only trained a top layer to perform classification on Chest X-ray (Wang et al., 2017) data set. And as seen in figure 2, we can observe the location where the model focuses on for classification. And each of the saliency mapping methods (and it’s smoother version) gives consistent results, indicating that the left bottom region of the image is a critical region.
Due to restrictions related to medical imaging data, it is not realistic or feasible to expect multiple health institution to share their data. However, thanks to increase in the demand of data scientist in many health organization it is reasonable to expect that each of those organization would be interested in applying a deep learning models as possible solutions. Hence we see the most optimal approach is to create a step by step tutorial series, either in a video format or a single web-page format, so that different organizations can implement this solution in their current system. Additionally, we are aiming to expand the example use-case of these methods, specifically, we are planning to use neural networks to achieve dual task on microscopic cell images. Not only we can perform segmentation of cell regions but also count the number of cells in a particular image, while outputting uncertainty for each measurements. Finally, possible future works includes, creating a proof of concept web application where medical professionals can upload their own data and see the promising result of this methodology.
This research was supported by Ryerson Vision lab, and the author of this paper wishes to express appreciations toward Dr. Neil Bruce and Dr. Kosta Derpanis for providing a comfortable lab space.
- Gal & Ghahramani (2016) Gal, Y. and Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of the 33rd International Conference on International Conference on Machine Learning - Volume 48, ICML’16, pp. 1050–1059. JMLR.org, 2016. URL http://dl.acm.org/citation.cfm?id=3045390.3045502.
- Ronneberger et al. (2015) Ronneberger, O., Fischer, P., and Brox, T. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234–241. Springer, 2015.
- Simonyan & Zisserman (2015) Simonyan, K. and Zisserman, A. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
- Simonyan et al. (2013) Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
- Springenberg et al. (2014) Springenberg, J. T., Dosovitskiy, A., Brox, T., and Riedmiller, M. Striving for simplicity: The all convolutional net. arXiv preprint arXiv:1412.6806, 2014.
- Staal et al. (2004) Staal, J., Abràmoff, M. D., Niemeijer, M., Viergever, M. A., and Van Ginneken, B. Ridge-based vessel segmentation in color images of the retina. IEEE transactions on medical imaging, 23(4):501–509, 2004.
- Sundararajan et al. (2017) Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pp. 3319–3328. JMLR. org, 2017.
- Wang et al. (2017) Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., and Summers, R. M. Chestx-ray8: Hospital-scale chest x-ray database and benchmarks on weakly-supervised classification and localization of common thorax diseases. In , July 2017.