Machine Learning on Biomedical Images: Interactive Learning, Transfer Learning, Class Imbalance, and Beyond

by   Naimul Mefraz Khan, et al.
Ryerson University

In this paper, we highlight three issues that limit performance of machine learning on biomedical images, and tackle them through 3 case studies: 1) Interactive Machine Learning (IML): we show how IML can drastically improve exploration time and quality of direct volume rendering. 2) transfer learning: we show how transfer learning along with intelligent pre-processing can result in better Alzheimer's diagnosis using a much smaller training set 3) data imbalance: we show how our novel focal Tversky loss function can provide better segmentation results taking into account the imbalanced nature of segmentation datasets. The case studies are accompanied by in-depth analytical discussion of results with possible future directions.



page 1

page 2

page 3

page 4


A Study of Data Pre-processing Techniques for Imbalanced Biomedical Data Classification

Biomedical data are widely accepted in developing prediction models for ...

UCAM Biomedical translation at WMT19: Transfer learning multi-domain ensembles

The 2019 WMT Biomedical translation task involved translating Medline ab...

A Study imbalance handling by various data sampling methods in binary classification

The purpose of this research report is to present the our learning curve...

How Transferable are Neural Networks in NLP Applications?

Transfer learning is aimed to make use of valuable knowledge in a source...

Low-Shot Classification: A Comparison of Classical and Deep Transfer Machine Learning Approaches

Despite the recent success of deep transfer learning approaches in NLP, ...

Deep Metric Learning Model for Imbalanced Fault Diagnosis

Intelligent diagnosis method based on data-driven and deep learning is a...

It's easy to fool yourself: Case studies on identifying bias and confounding in bio-medical datasets

Confounding variables are a well known source of nuisance in biomedical ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.


  • [1] N. Abraham and N. M. Khan. A novel focal tversky loss function with improved attention u-net for segmentation. In IEEE Symposium on Biomedical Imaging, 2019.
  • [2] A. Gupta, M. Ayhan, and A. Maida. Natural image bases to represent neuroimaging data. In International Conference on Machine Learning, 2013.
  • [3] S. R. Hashemi et al. Tversky as a loss function for highly unbalanced image segmentation using 3d fully convolutional deep networks. CoRR, abs/1803.11078, 2018.
  • [4] M. Hon and N. M. Khan. Towards alzheimer’s disease classification through transfer learning. In IEEE International Conference on Bioinformatics and Biomedicine, 2017.
  • [5] Hosseini-Asl et al. Alzheimer’s disease diagnostics by adaptation of 3d convolutional network. In IEEE International Conference Image Processing, 2016.
  • [6] C. R. Jack et al. The alzheimer’s disease neuroimaging initiative (adni): Mri methods. Journal of magnetic resonance imaging, 27(4):685–691, 2008.
  • [7] N. M. Khan, M. Kyan, and L. Guan. Intuitive volume exploration through spherical self-organizing map and color harmonization. Neurocomputing, 147:160–173, 2015.
  • [8] B. Kim. Interactive and interpretable machine learning models for human machine collaboration. PhD thesis, Massachusetts Institute of Technology, 2015.
  • [9] G. Kindlmann and J. Durkin. Semi-automatic generation of transfer functions for direct volume rendering. In IEEE symposium on Volume visualization, 1998.
  • [10] S. Klöppel et al. Accuracy of dementia diagnosis - a direct comparison between radiologists and a computerized method. Brain, 131(11):2969–2974, 2008.
  • [11] T. Kohonen, editor. Self-organizing maps. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 1997.
  • [12] B. C. Kwon et al.

    Retainvis: Visual analytics with interpretable and interactive recurrent neural networks on electronic medical records.

    IEEE transactions on visualization and computer graphics, 25(1):299–309, 2019.
  • [13] T.-Y. Lin, P. Goyal, R. Girshick, K. He, and P. Dollár. Focal loss for dense object detection. IEEE transactions on pattern analysis and machine intelligence, 2018.
  • [14] Y. Liu, C. Lisle, and J. Collins. Quick2insight: A user-friendly framework for interactive rendering of biological image volumes. In

    IEEE Symposium on Biological Data Visualization

    . IEEE Press, 2011.
  • [15] R. Maciejewski, I. Wu, W. Chen, , and D. Ebert. Structuring feature space: A non-parametric method for volumetric transfer function generation. IEEE Transactions on Visualization and Computer Graphics, 15:1473–1480, 2009.
  • [16] B. Nguyen et al. A clustering-based system to automate transfer function design for medical image visualization. The Visual Computer, 28:181–191, 2012.
  • [17] O. Oktay et al. Attention u-net: Learning where to look for the pancreas. arXiv preprint arXiv:1804.03999, 2018.
  • [18] C. Plant et al. Automated detection of brain atrophy patterns based on mri for the prediction of alzheimer’s disease. Neuroimage, 50(1):162–174, 2010.
  • [19] S. Roettger. The volume library. 2012.
  • [20] S. Roettger, M. Bauer, and M. Stamminger. Spatialized transfer functions. In IEEE/Eurographics symposium on visualization. IEEE Press, 2005.
  • [21] O. Ronneberger, P. Fischer, and T. Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2015.
  • [22] O. Russakovsky et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):211–252, 2015.
  • [23] A. Sangole. Data-Driven Modeling Using Spherical Self-Organizing Maps. PhD thesis, Western University, 2002.
  • [24] S. Sarraf, G. Tofighi, et al. Deepad: Alzheimer’s disease classification via deep convolutional neural networks using mri and fmri. bioRxiv, page 070441, 2016.
  • [25] M. Selver et al.

    Semiautomatic transfer function initialization for abdominal visualization using self-generating hierarchical radial basis function.

    IEEE Transactions on Visualization and Computer Graphics, 15:395–409, May 2009.
  • [26] K. Simonyan and A. Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
  • [27] C. Studholme, D. L. Hill, and D. J. Hawkes. An overlap invariant entropy measure of 3d medical image alignment. Pattern recognition, 32(1):71–86, 1999.
  • [28] C. H. Sudre et al. Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In Deep Learning in Medical Image Analysis for Clinical Decision Support, pages 240–248. Springer, 2017.
  • [29] N. Tajbakhsh et al. Convolutional neural networks for medical image analysis: Full training or fine tuning? IEEE transactions on medical imaging, 35(5):1299–1312, 2016.
  • [30] K. C. Wong et al. 3d segmentation with exponential logarithmic loss for highly unbalanced object sizes. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 2018.
  • [31] M. H. Yap et al. Automated breast ultrasound lesions detection using convolutional neural networks. IEEE journal of biomedical and health informatics, 22(4):1218–1226, 2018.
  • [32] J. Yosinski, J. Clune, Y. Bengio, and H. Lipson. How transferable are features in deep neural networks? In Advances in neural information processing systems, 2014.