Multimodality Biomedical Image Registration using Free Point Transformer Networks

by   Zachary M. C. Baum, et al.

We describe a point-set registration algorithm based on a novel free point transformer (FPT) network, designed for points extracted from multimodal biomedical images for registration tasks, such as those frequently encountered in ultrasound-guided interventional procedures. FPT is constructed with a global feature extractor which accepts unordered source and target point-sets of variable size. The extracted features are conditioned by a shared multilayer perceptron point transformer module to predict a displacement vector for each source point, transforming it into the target space. The point transformer module assumes no vicinity or smoothness in predicting spatial transformation and, together with the global feature extractor, is trained in a data-driven fashion with an unsupervised loss function. In a multimodal registration task using prostate MR and sparsely acquired ultrasound images, FPT yields comparable or improved results over other rigid and non-rigid registration methods. This demonstrates the versatility of FPT to learn registration directly from real, clinical training data and to generalize to a challenging task, such as the interventional application presented.



page 1

page 2

page 3

page 4


Real-time multimodal image registration with partial intraoperative point-set data

We present Free Point Transformer (FPT) - a deep neural network architec...

Adversarial Image Registration with Application for MR and TRUS Image Fusion

Robust and accurate alignment of multimodal medical images is a very cha...

Coherent Point Drift Networks: Unsupervised Learning of Non-Rigid Point Set Registration

Given new pairs of source and target point sets, standard point set regi...

Astroalign: A Python module for astronomical image registration

We present an algorithm implemented in the astroalign Python module for ...

RGB-Topography and X-rays Image Registration for Idiopathic Scoliosis Children Patient Follow-up

Children diagnosed with a scoliosis pathology are exposed during their f...

End-to-end Ultrasound Frame to Volume Registration

Fusing intra-operative 2D transrectal ultrasound (TRUS) image with pre-o...

A registration error estimation framework for correlative imaging

Correlative imaging workflows are now widely used in bioimaging and aims...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.