Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration

11/16/2019
by   Robert Grupp, et al.
26

Fluoroscopy is the standard imaging modality used to guide hip surgery and is therefore a natural sensor for computer-assisted navigation. In order to efficiently solve the complex registration problems presented during navigation, human-assisted annotations of the intraoperative image are typically required. This manual initialization interferes with the surgical workflow and diminishes any advantages gained from navigation. We propose a method for fully automatic registration using annotations produced by a neural network. Neural networks are trained to simultaneously segment anatomy and identify landmarks in fluoroscopy. Training data is obtained using an intraoperatively incompatible 2D/3D registration of hip anatomy. Ground truth 2D labels are established using projected 3D annotations. Intraoperative registration couples an intensity-based strategy with annotations inferred by the network and requires no human assistance. Ground truth labels were obtained in 366 fluoroscopic images across 6 cadaveric specimens. In a leave-one-subject-out experiment, networks obtained mean dice coefficients for left and right hemipelves, left and right femurs of 0.86, 0.87, 0.90, and 0.84. The mean 2D landmark error was 5.0 mm. The pelvis was registered within 1 degree for 86 with an average runtime of 7 seconds. In comparison, an intensity-only approach without manual initialization, registered the pelvis to 1 degree in 18 images. We have created the first accurately annotated, non-synthetic, dataset of hip fluoroscopy. By using these annotations as training data for neural networks, state of the art performance in fluoroscopic segmentation and landmark localization was achieved. Integrating these annotations allows for a robust, fully automatic, and efficient intraoperative registration during fluoroscopic navigation of the hip.

READ FULL TEXT

page 3

page 5

page 8

page 14

04/18/2019

Fully Automatic Segmentation of 3D Brain Ultrasound: Learning from Coarse Annotations

Intra-operative ultrasound is an increasingly important imaging modality...
01/20/2019

Localizing dexterous surgical tools in X-ray for image-based navigation

X-ray image based surgical tool navigation is fast and supplies accurate...
09/23/2019

Patch-Based Image Similarity for Intraoperative 2D/3D Pelvis Registration During Periacetabular Osteotomy

Periacetabular osteotomy is a challenging surgical procedure for treatin...
12/31/2019

BIRL: Benchmark on Image Registration methods with Landmark validation

This report presents a generic image registration benchmark with automat...
11/14/2020

Pose-dependent weights and Domain Randomization for fully automatic X-ray to CT Registration

Fully automatic X-ray to CT registration requires a solid initialization...
06/24/2021

High-resolution Image Registration of Consecutive and Re-stained Sections in Histopathology

We compare variational image registration in consectutive and re-stained...
09/17/2017

Automatic Tool Landmark Detection for Stereo Vision in Robot-Assisted Retinal Surgery

Computer vision and robotics are being increasingly applied in medical i...

Code Repositories

DeepFluoroLabeling-IPCAI2020

Code and data for the "annotation" component of the IPCAI 2020 paper: "Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration." https://arxiv.org/abs/1911.07042 or https://doi.org/10.1007/s11548-020-02162-7


view repo

Regi2D3D-IPCAI2020

Code for the registration component of the IPCAI 2020 paper: "Automatic Annotation of Hip Anatomy in Fluoroscopy for Robust and Efficient 2D/3D Registration." https://arxiv.org/abs/1911.07042 or https://doi.org/10.1007/s11548-020-02162-7


view repo