Interactive Medical Image Segmentation using Deep Learning with Image-specific Fine-tuning

by   Guotai Wang, et al.

Convolutional neural networks (CNNs) have achieved state-of-the-art performance for automatic medical image segmentation. However, they have not demonstrated sufficiently accurate and robust results for clinical use. In addition, they are limited by the lack of image-specific adaptation and the lack of generalizability to previously unseen object classes. To address these problems, we propose a novel deep learning-based framework for interactive segmentation by incorporating CNNs into a bounding box and scribble-based segmentation pipeline. We propose image-specific fine-tuning to make a CNN model adaptive to a specific test image, which can be either unsupervised (without additional user interactions) or supervised (with additional scribbles). We also propose a weighted loss function considering network and interaction-based uncertainty for the fine-tuning. We applied this framework to two applications: 2D segmentation of multiple organs from fetal MR slices, where only two types of these organs were annotated for training; and 3D segmentation of brain tumor core (excluding edema) and whole brain tumor (including edema) from different MR sequences, where only tumor cores in one MR sequence were annotated for training. Experimental results show that 1) our model is more robust to segment previously unseen objects than state-of-the-art CNNs; 2) image-specific fine-tuning with the proposed weighted loss function significantly improves segmentation accuracy; and 3) our method leads to accurate results with fewer user interactions and less user time than traditional interactive segmentation methods.


page 3

page 5

page 6

page 7

page 8

page 9


DeepIGeoS: A Deep Interactive Geodesic Framework for Medical Image Segmentation

Accurate medical image segmentation is essential for diagnosis, surgical...

Learning to Learn Unlearned Feature for Brain Tumor Segmentation

We propose a fine-tuning algorithm for brain tumor segmentation that nee...

Interactive Segmentation via Deep Learning and B-Spline Explicit Active Surfaces

Automatic medical image segmentation via convolutional neural networks (...

Interactive Medical Image Segmentation via Point-Based Interaction and Sequential Patch Learning

Due to low tissue contrast, irregular object appearance, and unpredictab...

DINs: Deep Interactive Networks for Neurofibroma Segmentation in Neurofibromatosis Type 1 on Whole-Body MRI

Neurofibromatosis type 1 (NF1) is an autosomal dominant tumor predisposi...

Deep Interactive Object Selection

Interactive object selection is a very important research problem and ha...

CAN3D: Fast 3D Medical Image Segmentation via Compact Context Aggregation

Direct automatic segmentation of objects from 3D medical imaging, such a...

Please sign up or login with your details

Forgot password? Click here to reset