Masked Co-attentional Transformer reconstructs 100x ultra-fast/low-dose whole-body PET from longitudinal images and anatomically guided MRI

by   Yan-Ran, et al.

Despite its tremendous value for the diagnosis, treatment monitoring and surveillance of children with cancer, whole body staging with positron emission tomography (PET) is time consuming and associated with considerable radiation exposure. 100x (1 whole-body PET reconstruction has the potential for cancer imaging with unprecedented speed and improved safety, but it cannot be achieved by the naive use of machine learning techniques. In this study, we utilize the global similarity between baseline and follow-up PET and magnetic resonance (MR) images to develop Masked-LMCTrans, a longitudinal multi-modality co-attentional CNN-Transformer that provides interaction and joint reasoning between serial PET/MRs of the same patient. We mask the tumor area in the referenced baseline PET and reconstruct the follow-up PET scans. In this manner, Masked-LMCTrans reconstructs 100x almost-zero radio-exposure whole-body PET that was not possible before. The technique also opens a new pathway for longitudinal radiology imaging reconstruction, a significantly under-explored area to date. Our model was trained and tested with Stanford PET/MRI scans of pediatric lymphoma patients and evaluated externally on PET/MRI images from Tübingen University. The high image quality of the reconstructed 100x whole-body PET images resulting from the application of Masked-LMCTrans will substantially advance the development of safer imaging approaches and shorter exam-durations for pediatric patients, as well as expand the possibilities for frequent longitudinal monitoring of these patients by PET.


page 1

page 5

page 6

page 7

page 9

page 17

page 19


Compressed sensing for longitudinal MRI: An adaptive-weighted approach

Purpose: Repeated brain MRI scans are performed in many clinical scenari...

Towards Lower-Dose PET using Physics-Based Uncertainty-Aware Multimodal Learning with Robustness to Out-of-Distribution Data

Radiation exposure in positron emission tomography (PET) imaging limits ...

3T MR-Guided Brachytherapy for Gynecologic Malignancies

Gynecologic malignancies are a leading cause of death in women worldwide...

DeepRec: A deep encoder-decoder network for directly solving the PET reconstruction inverse problem

Positron emission tomography (PET) is a cornerstone of modern radiology....

A Deep Learning Approach Using Masked Image Modeling for Reconstruction of Undersampled K-spaces

Magnetic Resonance Imaging (MRI) scans are time consuming and precarious...

Please sign up or login with your details

Forgot password? Click here to reset