Dilated FCN for Multi-Agent 2D/3D Medical Image Registration

11/22/2017
by   Shun Miao, et al.
0

2D/3D image registration to align a 3D volume and 2D X-ray images is a challenging problem due to its ill-posed nature and various artifacts presented in 2D X-ray images. In this paper, we propose a multi-agent system with an auto attention mechanism for robust and efficient 2D/3D image registration. Specifically, an individual agent is trained with dilated Fully Convolutional Network (FCN) to perform registration in a Markov Decision Process (MDP) by observing a local region, and the final action is then taken based on the proposals from multiple agents and weighted by their corresponding confidence levels. The contributions of this paper are threefold. First, we formulate 2D/3D registration as a MDP with observations, actions, and rewards properly defined with respect to X-ray imaging systems. Second, to handle various artifacts in 2D X-ray images, multiple local agents are employed efficiently via FCN-based structures, and an auto attention mechanism is proposed to favor the proposals from regions with more reliable visual cues. Third, a dilated FCN-based training mechanism is proposed to significantly reduce the Degree of Freedom in the simulation of registration environment, and drastically improve training efficiency by an order of magnitude compared to standard CNN-based training method. We demonstrate that the proposed method achieves high robustness on both spine cone beam Computed Tomography data with a low signal-to-noise ratio and data from minimally invasive spine surgery where severe image artifacts and occlusions are presented due to metal screws and guide wires, outperforming other state-of-the-art methods (single agent-based and optimization-based) by a large margin.

READ FULL TEXT

page 2

page 5

research
11/30/2016

An Artificial Agent for Robust Image Registration

3-D image registration, which involves aligning two or more images, is a...
research
08/04/2018

Troy: Give Attention to Saliency and for Saliency

Saliency detection or foreground segmentation is a fundamental and impor...
research
10/31/2017

Segmentation-by-Detection: A Cascade Network for Volumetric Medical Image Segmentation

We propose an attention mechanism for 3D medical image segmentation. The...
research
02/01/2022

A training-free recursive multiresolution framework for diffeomorphic deformable image registration

Diffeomorphic deformable image registration is one of the crucial tasks ...
research
03/09/2020

Restore from Restored: Video Restoration with Pseudo Clean Video

In this paper, we propose a self-supervised video denoising method calle...
research
12/15/2020

Representing Ambiguity in Registration Problems with Conditional Invertible Neural Networks

Image registration is the basis for many applications in the fields of m...

Please sign up or login with your details

Forgot password? Click here to reset