Multi-Domain Image Completion for Random Missing Input Data

by   Liyue Shen, et al.
Stanford University

Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e.g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI). However, due to possible data corruption and different imaging protocols, the availability of images for each domain could vary amongst multiple data sources in practice, which makes it challenging to build a universal model with a varied set of input data. To tackle this problem, we propose a general approach to complete the random missing domain(s) data in real applications. Specifically, we develop a novel multi-domain image completion method that utilizes a generative adversarial network (GAN) with a representational disentanglement scheme to extract shared skeleton encoding and separate flesh encoding across multiple domains. We further illustrate that the learned representation in multi-domain image completion could be leveraged for high-level tasks, e.g., segmentation, by introducing a unified framework consisting of image completion and segmentation with a shared content encoder. The experiments demonstrate consistent performance improvement on three datasets for brain tumor segmentation, prostate segmentation, and facial expression image completion respectively.


page 3

page 11

page 26

page 27

page 28

page 29

page 30

page 31


Brain Tumor Segmentation on MRI with Missing Modalities

Brain Tumor Segmentation from magnetic resonance imaging (MRI) is a crit...

The Brain Tumor Segmentation (BraTS) Challenge 2023: Brain MR Image Synthesis for Tumor Segmentation (BraSyn)

Automated brain tumor segmentation methods are well established, reachin...

M3AE: Multimodal Representation Learning for Brain Tumor Segmentation with Missing Modalities

Multimodal magnetic resonance imaging (MRI) provides complementary infor...

Brain tumor segmentation with missing modalities via latent multi-source correlation representation

Multimodal MR images can provide complementary information for accurate ...

Modality Completion via Gaussian Process Prior Variational Autoencoders for Multi-Modal Glioma Segmentation

In large studies involving multi protocol Magnetic Resonance Imaging (MR...

Disentanglement enables cross-domain Hippocampus Segmentation

Limited amount of labelled training data are a common problem in medical...

Exploiting Partial Common Information Microstructure for Multi-Modal Brain Tumor Segmentation

Learning with multiple modalities is crucial for automated brain tumor s...

Please sign up or login with your details

Forgot password? Click here to reset