Real-time multimodal image registration with partial intraoperative point-set data

09/10/2021
by   Zachary M. C. Baum, et al.
6

We present Free Point Transformer (FPT) - a deep neural network architecture for non-rigid point-set registration. Consisting of two modules, a global feature extraction module and a point transformation module, FPT does not assume explicit constraints based on point vicinity, thereby overcoming a common requirement of previous learning-based point-set registration methods. FPT is designed to accept unordered and unstructured point-sets with a variable number of points and uses a "model-free" approach without heuristic constraints. Training FPT is flexible and involves minimizing an intuitive unsupervised loss function, but supervised, semi-supervised, and partially- or weakly-supervised training are also supported. This flexibility makes FPT amenable to multimodal image registration problems where the ground-truth deformations are difficult or impossible to measure. In this paper, we demonstrate the application of FPT to non-rigid registration of prostate magnetic resonance (MR) imaging and sparsely-sampled transrectal ultrasound (TRUS) images. The registration errors were 4.71 mm and 4.81 mm for complete TRUS imaging and sparsely-sampled TRUS imaging, respectively. The results indicate superior accuracy to the alternative rigid and non-rigid registration algorithms tested and substantially lower computation time. The rapid inference possible with FPT makes it particularly suitable for applications where real-time registration is beneficial.

READ FULL TEXT

Authors

page 8

page 11

page 14

page 16

page 17

08/05/2020

Multimodality Biomedical Image Registration using Free Point Transformer Networks

We describe a point-set registration algorithm based on a novel free poi...
07/14/2021

End-to-end Ultrasound Frame to Volume Registration

Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-o...
07/09/2020

Prostate motion modelling using biomechanically-trained deep neural networks on unstructured nodes

In this paper, we propose to train deep neural networks with biomechanic...
07/11/2018

Temporal Convolution Networks for Real-Time Abdominal Fetal Aorta Analysis with Ultrasound

The automatic analysis of ultrasound sequences can substantially improve...
03/10/2021

A registration error estimation framework for correlative imaging

Correlative imaging workflows are now widely used in bioimaging and aims...
09/06/2019

Astroalign: A Python module for astronomical image registration

We present an algorithm implemented in the astroalign Python module for ...
11/05/2017

Label-driven weakly-supervised learning for multimodal deformable image registration

Spatially aligning medical images from different modalities remains a ch...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.