Towards Cross-modality Medical Image Segmentation with Online Mutual Knowledge Distillation

by   Kang Li, et al.

The success of deep convolutional neural networks is partially attributed to the massive amount of annotated training data. However, in practice, medical data annotations are usually expensive and time-consuming to be obtained. Considering multi-modality data with the same anatomic structures are widely available in clinic routine, in this paper, we aim to exploit the prior knowledge (e.g., shape priors) learned from one modality (aka., assistant modality) to improve the segmentation performance on another modality (aka., target modality) to make up annotation scarcity. To alleviate the learning difficulties caused by modality-specific appearance discrepancy, we first present an Image Alignment Module (IAM) to narrow the appearance gap between assistant and target modality data.We then propose a novel Mutual Knowledge Distillation (MKD) scheme to thoroughly exploit the modality-shared knowledge to facilitate the target-modality segmentation. To be specific, we formulate our framework as an integration of two individual segmentors. Each segmentor not only explicitly extracts one modality knowledge from corresponding annotations, but also implicitly explores another modality knowledge from its counterpart in mutual-guided manner. The ensemble of two segmentors would further integrate the knowledge from both modalities and generate reliable segmentation results on target modality. Experimental results on the public multi-class cardiac segmentation data, i.e., MMWHS 2017, show that our method achieves large improvements on CT segmentation by utilizing additional MRI data and outperforms other state-of-the-art multi-modality learning methods.


page 3

page 6

page 7


Unpaired Multi-modal Segmentation via Knowledge Distillation

Multi-modal learning is typically performed with network architectures c...

Statistical modality tagging from rule-based annotations and crowdsourcing

We explore training an automatic modality tagger. Modality is the attitu...

Dual-Teacher: Integrating Intra-domain and Inter-domain Teachers for Annotation-efficient Cardiac Segmentation

Medical image annotations are prohibitively time-consuming and expensive...

An improved 3D region detection network: automated detection of the 12th thoracic vertebra in image guided radiation therapy

Abstract. Image guidance has been widely used in radiation therapy. Corr...

Graph Flow: Cross-layer Graph Flow Distillation for Dual-Efficient Medical Image Segmentation

With the development of deep convolutional neural networks, medical imag...

Mind The Gap: Alleviating Local Imbalance for Unsupervised Cross-Modality Medical Image Segmentation

Unsupervised cross-modality medical image adaptation aims to alleviate t...

Challenge-Aware RGBT Tracking

RGB and thermal source data suffer from both shared and specific challen...