Learning Multi-Modal Brain Tumor Segmentation from Privileged Semi-Paired MRI Images with Curriculum Disentanglement Learning

08/26/2022
by   Zecheng Liu, et al.
0

Due to the difficulties of obtaining multimodal paired images in clinical practice, recent studies propose to train brain tumor segmentation models with unpaired images and capture complementary information through modality translation. However, these models cannot fully exploit the complementary information from different modalities. In this work, we thus present a novel two-step (intra-modality and inter-modality) curriculum disentanglement learning framework to effectively utilize privileged semi-paired images, i.e. limited paired images that are only available in training, for brain tumor segmentation. Specifically, in the first step, we propose to conduct reconstruction and segmentation with augmented intra-modality style-consistent images. In the second step, the model jointly performs reconstruction, unsupervised/supervised translation, and segmentation for both unpaired and paired inter-modality images. A content consistency loss and a supervised translation loss are proposed to leverage complementary information from different modalities in this step. Through these two steps, our method effectively extracts modality-specific style codes describing the attenuation of tissue features and image contrast, and modality-invariant content codes containing anatomical and functional information from the input images. Experiments on three brain tumor segmentation tasks show that our model outperforms competing segmentation models based on unpaired images.

READ FULL TEXT

page 1

page 2

page 3

page 7

research
07/08/2019

Unified Attentional Generative Adversarial Network for Brain Tumor Segmentation From Multimodal Unpaired Images

In medical applications, the same anatomical structures may be observed ...
research
02/22/2020

Robust Multimodal Brain Tumor Segmentation via Feature Disentanglement and Gated Fusion

Accurate medical image segmentation commonly requires effective learning...
research
10/19/2020

Modality-Pairing Learning for Brain Tumor Segmentation

Automatic brain tumor segmentation from multi-modality Magnetic Resonanc...
research
01/14/2021

A Unified Conditional Disentanglement Framework for Multimodal Brain MR Image Translation

Multimodal MRI provides complementary and clinically relevant informatio...
research
12/14/2022

M-GenSeg: Domain Adaptation For Target Modality Tumor Segmentation With Annotation-Efficient Supervision

Automated medical image segmentation using deep neural networks typicall...
research
12/09/2019

Semi-supervised Learning Approach to Generate Neuroimaging Modalities with Adversarial Training

Magnetic Resonance Imaging (MRI) of the brain can come in the form of di...
research
02/14/2022

A Survey of Cross-Modality Brain Image Synthesis

The existence of completely aligned and paired multi-modal neuroimaging ...

Please sign up or login with your details

Forgot password? Click here to reset