Self-Attentive Spatial Adaptive Normalization for Cross-Modality Domain Adaptation

by   Devavrat Tomar, et al.

Despite the successes of deep neural networks on many challenging vision tasks, they often fail to generalize to new test domains that are not distributed identically to the training data. The domain adaptation becomes more challenging for cross-modality medical data with a notable domain shift. Given that specific annotated imaging modalities may not be accessible nor complete. Our proposed solution is based on the cross-modality synthesis of medical images to reduce the costly annotation burden by radiologists and bridge the domain gap in radiological images. We present a novel approach for image-to-image translation in medical images, capable of supervised or unsupervised (unpaired image data) setups. Built upon adversarial training, we propose a learnable self-attentive spatial normalization of the deep convolutional generator network's intermediate activations. Unlike previous attention-based image-to-image translation approaches, which are either domain-specific or require distortion of the source domain's structures, we unearth the importance of the auxiliary semantic information to handle the geometric changes and preserve anatomical structures during image translation. We achieve superior results for cross-modality segmentation between unpaired MRI and CT data for multi-modality whole heart and multi-modal brain tumor MRI (T1/T2) datasets compared to the state-of-the-art methods. We also observe encouraging results in cross-modality conversion for paired MRI and CT images on a brain dataset. Furthermore, a detailed analysis of the cross-modality image translation, thorough ablation studies confirm our proposed method's efficacy.


page 1

page 5

page 7

page 9

page 10

page 11

page 13


MRI Image-to-Image Translation for Cross-Modality Image Registration and Segmentation

We develop a novel cross-modality generation framework that learns to ge...

Disentangling A Single MR Modality

Disentangling anatomical and contrast information from medical images ha...

PnP-AdaNet: Plug-and-Play Adversarial Domain Adaptation Network with a Benchmark at Cross-modality Cardiac Segmentation

Deep convolutional networks have demonstrated the state-of-the-art perfo...

M-GenSeg: Domain Adaptation For Target Modality Tumor Segmentation With Annotation-Efficient Supervision

Automated medical image segmentation using deep neural networks typicall...

Image-level supervision and self-training for transformer-based cross-modality tumor segmentation

Deep neural networks are commonly used for automated medical image segme...

Unified cross-modality feature disentangler for unsupervised multi-domain MRI abdomen organs segmentation

Our contribution is a unified cross-modality feature disentagling approa...

Cross-Domain Medical Image Translation by Shared Latent Gaussian Mixture Model

Current deep learning based segmentation models often generalize poorly ...

Please sign up or login with your details

Forgot password? Click here to reset