The entire network structure of Crossmodal Transformer

04/29/2021
by   Meng Li, et al.
0

Since the mapping relationship between definitized intra-interventional 2D X-ray and undefined pre-interventional 3D Computed Tomography(CT) is uncertain, auxiliary positioning devices or body markers, such as medical implants, are commonly used to determine this relationship. However, such approaches can not be widely used in clinical due to the complex realities. To determine the mapping relationship, and achieve a initializtion post estimation of human body without auxiliary equipment or markers, a cross-modal matching transformer network is proposed to matching 2D X-ray and 3D CT images directly. The proposed approach first deep learns skeletal features from 2D X-ray and 3D CT images. The features are then converted into 1D X-ray and CT representation vectors, which are combined using a multi-modal transformer. As a result, the well-trained network can directly predict the spatial correspondence between arbitrary 2D X-ray and 3D CT. The experimental results show that when combining our approach with the conventional approach, the achieved accuracy and speed can meet the basic clinical intervention needs, and it provides a new direction for intra-interventional registration.

READ FULL TEXT

page 3

page 4

research
05/31/2023

XTransCT: Ultra-Fast Volumetric CT Reconstruction using Two Orthogonal X-Ray Projections via a Transformer Network

Computed tomography (CT) scans offer a detailed, three-dimensional repre...
research
11/09/2020

Patch-based field-of-view matching in multi-modal images for electroporation-based ablations

Various multi-modal imaging sensors are currently involved at different ...
research
05/26/2022

Deep-XFCT: Deep learning 3D-mineral liberation analysis with micro X-ray fluorescence and computed tomography

The rapid development of X-ray micro-computed tomography (micro-CT) open...
research
03/09/2023

Perspective Projection-Based 3D CT Reconstruction from Biplanar X-rays

X-ray computed tomography (CT) is one of the most common imaging techniq...
research
11/10/2020

Pristine annotations-based multi-modal trained artificial intelligence solution to triage chest X-ray for COVID-19

The COVID-19 pandemic continues to spread and impact the well-being of t...
research
08/08/2022

SLAM-TKA: Real-time Intra-operative Measurement of Tibial Resection Plane in Conventional Total Knee Arthroplasty

Total knee arthroplasty (TKA) is a common orthopaedic surgery to replace...
research
04/04/2023

EduceLab-Scrolls: Verifiable Recovery of Text from Herculaneum Papyri using X-ray CT

We present a complete software pipeline for revealing the hidden texts o...

Please sign up or login with your details

Forgot password? Click here to reset