Holographic Visualisation of Radiology Data and Automated Machine Learning-based Medical Image Segmentation

08/15/2018
by   Lucian Trestioreanu, et al.
8

Within this thesis we propose a platform for combining Augmented Reality (AR) hardware with machine learning in a user-oriented pipeline, offering to the medical staff an intuitive 3D visualization of volumetric Computed Tomography (CT) and Magnetic Resonance Imaging (MRI) medical image segmentations inside the AR headset, that does not need human intervention for loading, processing and segmentation of medical images. The AR visualization, based on Microsoft HoloLens, employs a modular and thus scalable frontend-backend architecture for real-time visualizations on multiple AR headsets. As Convolutional Neural Networks (CNNs) have lastly demonstrated superior performance for the machine learning task of image semantic segmentation, the pipeline also includes a fully automated CNN algorithm for the segmentation of the liver from CT scans. The model is based on the Deep Retinal Image Understanding (DRIU) model which is a Fully Convolutional Network with side outputs from feature maps with different resolution, extracted at different stages of the network. The algorithm is 2.5D which means that the input is a set of consecutive scan slices. The experiments have been performed on the Liver Tumor Segmentation Challenge (LiTS) dataset for liver segmentation and demonstrated good results and flexibility. While multiple approaches exist in the domain, only few of them have focused on overcoming the practical aspects which still largely hold this technology away from the operating rooms. In line with this, we also are next planning an evaluation from medical doctors and radiologists in a real-world environment.

READ FULL TEXT

page 25

page 26

page 27

page 31

page 32

page 36

page 38

page 39

research
03/23/2018

Deep learning and its application to medical image segmentation

One of the most common tasks in medical imaging is semantic segmentation...
research
07/19/2022

Large-Kernel Attention for 3D Medical Image Segmentation

Automatic segmentation of multiple organs and tumors from 3D medical ima...
research
06/10/2022

ClamNet: Using contrastive learning with variable depth Unets for medical image segmentation

Unets have become the standard method for semantic segmentation of medic...
research
06/11/2019

`Project & Excite' Modules for Segmentation of Volumetric Medical Scans

Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art pe...
research
05/22/2018

Knowledge-based Fully Convolutional Network and Its Application in Segmentation of Lung CT Images

A variety of deep neural networks have been applied in medical image seg...
research
08/07/2019

Kidney and Kidney Tumor Segmentation using a Logical Ensemble of U-nets with Volumetric Validation

Automated medical image segmentation is a priority research area for com...
research
02/25/2020

Recalibrating 3D ConvNets with Project Excite

Fully Convolutional Neural Networks (F-CNNs) achieve state-of-the-art pe...

Please sign up or login with your details

Forgot password? Click here to reset