Generative Adversarial Networks for MR-CT Deformable Image Registration

by   Christine Tanner, et al.

Deformable Image Registration (DIR) of MR and CT images is one of the most challenging registration task, due to the inherent structural differences of the modalities and the missing dense ground truth. Recently cycle Generative Adversarial Networks (cycle-GANs) have been used to learn the intensity relationship between these 2 modalities for unpaired brain data. Yet its usefulness for DIR was not assessed. In this study we evaluate the DIR performance for thoracic and abdominal organs after synthesis by cycle-GAN. We show that geometric changes, which differentiate the two populations (e.g. inhale vs. exhale), are readily synthesized as well. This causes substantial problems for any application which relies on spatial correspondences being preserved between the real and the synthesized image (e.g. plan, segmentation, landmark propagation). To alleviate this problem, we investigated reducing the spatial information provided to the discriminator by decreasing the size of its receptive fields. Image synthesis was learned from 17 unpaired subjects per modality. Registration performance was evaluated with respect to manual segmentations of 11 structures for 3 subjects from the VISERAL challenge. State-of-the-art DIR methods based on Normalized Mutual Information (NMI), Modality Independent Neighborhood Descriptor (MIND) and their novel combination achieved a mean segmentation overlap ratio of 76.7, 67.7, 76.9 69.1 correlation, due to the poor performance on the thoracic region, where large lung volume changes were synthesized. Performance for the abdominal region was similar to that of CT-MRI NMI registration (77.4 vs. 78.8 synthesizing MRIs (12 slices) and medium sized receptive fields for the discriminator.


page 9

page 10


Multi-Modality Generative Adversarial Networks with Tumor Consistency Loss for Brain MR Image Synthesis

Magnetic Resonance (MR) images of different modalities can provide compl...

Synthesis and Inpainting-Based MR-CT Registration for Image-Guided Thermal Ablation of Liver Tumors

Thermal ablation is a minimally invasive procedure for treat-ing small o...

Adversarial Synthesis Learning Enables Segmentation Without Target Modality Ground Truth

A lack of generalizability is one key limitation of deep learning based ...

MRI Image-to-Image Translation for Cross-Modality Image Registration and Segmentation

We develop a novel cross-modality generation framework that learns to ge...

Unsupervised learning for cross-domain medical image synthesis using deformation invariant cycle consistency networks

Recently, the cycle-consistent generative adversarial networks (CycleGAN...

Quantification of Local Metabolic Tumor Volume Changes by Registering Blended PET-CT Images for Prediction of Pathologic Tumor Response

Quantification of local metabolic tumor volume (MTV) chan-ges after Chem...

Communal Domain Learning for Registration in Drifted Image Spaces

Designing a registration framework for images that do not share the same...

Please sign up or login with your details

Forgot password? Click here to reset