Weakly-Supervised Convolutional Neural Networks for Multimodal Image Registration

by   Yipeng Hu, et al.

One of the fundamental challenges in supervised learning for multimodal image registration is the lack of ground-truth for voxel-level spatial correspondence. This work describes a method to infer voxel-level transformation from higher-level correspondence information contained in anatomical labels. We argue that such labels are more reliable and practical to obtain for reference sets of image pairs than voxel-level correspondence. Typical anatomical labels of interest may include solid organs, vessels, ducts, structure boundaries and other subject-specific ad hoc landmarks. The proposed end-to-end convolutional neural network approach aims to predict displacement fields to align multiple labelled corresponding structures for individual image pairs during the training, while only unlabelled image pairs are used as the network input for inference. We highlight the versatility of the proposed strategy, for training, utilising diverse types of anatomical labels, which need not to be identifiable over all training image pairs. At inference, the resulting 3D deformable image registration algorithm runs in real-time and is fully-automated without requiring any anatomical labels or initialisation. Several network architecture variants are compared for registering T2-weighted magnetic resonance images and 3D transrectal ultrasound images from prostate cancer patients. A median target registration error of 3.6 mm on landmark centroids and a median Dice of 0.87 on prostate glands are achieved from cross-validation experiments, in which 108 pairs of multimodal images from 76 patients were tested with high-quality anatomical labels.


page 3

page 5

page 6

page 11

page 13

page 14


Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a ch...

Adversarial Deformation Regularization for Training Image Registration Neural Networks

We describe an adversarial learning approach to constrain convolutional ...

Conditional Segmentation in Lieu of Image Registration

Classical pairwise image registration methods search for a spatial trans...

Learning Multi-Modal Volumetric Prostate Registration with Weak Inter-Subject Spatial Correspondence

Recent studies demonstrated the eligibility of convolutional neural netw...

Spatial Correspondence between Graph Neural Network-Segmented Images

Graph neural networks (GNNs) have been proposed for medical image segmen...

Automatic Landmarks Correspondence Detection in Medical Images with an Application to Deformable Image Registration

Deformable Image Registration (DIR) can benefit from additional guidance...

Weakly Supervised Localisation for Fetal Ultrasound Images

This paper addresses the task of detecting and localising fetal anatomic...

Please sign up or login with your details

Forgot password? Click here to reset