Information extracted from MRI can be dramatically enhanced with the use of the exogenous contrast agent, gadolinium (Gd). Gd-enhanced MRI is routinely used to better visualize nearly all neurological disease - including strokes, tumors, infections, and neuroinflammation. Moreover, since steady-state Gd-enhanced MRI technique can generate high resolution maps of the cerebral blood volume (CBV) and cerebral blood flow (CBF), both tightly coupled to brain metabolism, Gd-enhanced MRI can be used as an fMRI tool [RN10]. More recently, because it generates CBV maps that are both quantitative and have high spatial resolution, this CBV-fMRI approach has been used to detect the earliest stages of Alzheimer’s disease [RN12] and schizophrenia [RN11], and to map the effects of normal aging [RN16].
Despite its significant advantages, reports of gadolinium retention in the brain and body after previous exposure to gadolinium based contrast agents (GBCAs) raise serious safety concerns in the clinical community [RN5]. Given the potential safety risks and the FDA warning [RN40], there is an urgent need to develop alternative imaging techniques that reduce the dose of Gd or eliminate it entirely to prevent Gd retention.
Recently, deep learning has shown a great potential to reduce Gd exposure in brain MRI [Gong]. This study shows that Gd dose can be reduced 10-fold while preserving contrast information and avoiding significant image quality degradation for visibility of pathology and delineation of brain lesions [Gong]. A recent study further demonstrates the possibility of predicting virtual Gd contrast in brain MRI from noncontrast multiparametric MRI scans [Can]
. The question remains whether it is feasible to estimate Gd contrast in brain MRI directly from a single noncontrast scan such as the most readily available structural MRI to ensure patient safety and imaging efficiency. We hypothesize that deep learning can produce Gd contrast in brain MRI directly from single noncontrast structural MRI. In the current study, we test this idea in mice as a proof of concept study before translating the same approach to humans. We utilize the residual attention U-Net architecture in our deep learning model to estimate Gd contrast from noncontrast T2-weighted (T2W) MRI for CBV mapping. Our method is evaluated for both wild-type (WT) mice and mice with Glioblastoma (GBM) at 9.4T.
2 Material and Methods
Experiments were performed following the National Institutes of Health (NIH) guidelines and were approved by the Institutional Animal Care and Use Committee (IACUC).
2.1 Animal Subject
Mice used in our study were divided into two groups: WT mice and mice with GBM. The WT group contained 49 healthy adult C576J/BL male mice scanned at 12-14 months old. The GBM group contained 10 adult C576J/BL male mice that were injected with PDGFB (+/+) PTEN (-/-) p53 (-/-) glioblastoma (GBM) cells [Lei]. 50,000 cells in 1 were stereotactically injected into the brain. MRI scans of GBM mice were obtained 10 days after injection.
2.2 MRI Acquisition and Preprocessing
For each mouse, T2W MRI scans were acquired using the 2D T2-weighted Turbo Rapid Acquisition with Refocused Echoes (RARE) sequence at 9.4T (i.e., TR/TE = 3500/45, RARE factor = 8, 76 in-plane resolution, 450 slice thickness; Bruker Biospec 94/30 USR equipped with CryoProbe).
As previously described, to derive the steady-state CBV maps, whole brain T2W MRI scans before (i.e., Pre) and 35 mins after (i.e., Post) IP injection of Gadodiamide at 10 are acquired with identical scan parameters (Fig 1.a) [RN9]. As a direct approach to reduce Gd dose, we also derive the 20% low-dose CBV maps as illustrated in Figure 1.b. For scans of tumor subjects, tumor masks are generated in addition to the brain masks using the Fuzzy-C-Means segmentation [RN30].
2.3 Deep Learning Model
2.3.1 Model Architecture
The deep learning architecture utilized in our study is the out-stand five-layer residual attention U-Net (ResAttU-Net) as illustrated in Figure 2
. This consists of a contraction path that encodes high-resolution data into low-resolution representations and an expansion path that decodes such encoded representations back to high-resolution images. We implemented our deep learning model using PyTorch framework with CUDA 10.0, 2 NVIDIA RTX 2080-TI GPUs and CentOS 6.
Both the encoder and decoder parts are based on the U-Net structure [RN44] where each stage consists of two series of 3
2 max-pooling for down sampling, while for the decoder part, four 22 upsampling layers convert low-resolution representation back to high resolution. Additionally, every stage of the decoder has concatenation from the encoder at the same level to give the model more accurate local information for assembling a precise prediction.
To further refine our network, as illustrated in Figure 2, the residual blocks and the attention gates are incorporated into the U-Net architecture. The residual block can optimize the signal and gradient propagation within a network while preventing overfitting from being a problem [RN45]. While the attention gate can significantly suppress feature responses in irrelevant background regions without needing to crop an ROI [article14].
2.3.2 Training the Model
We use ADAM as the optimizer with a learning rate of 10-3 and maximum 200 epochs with early stop. We used a batch size of 3 and randomize non-overlapping input images for training. Scans of 6 standalone mice were used for validation.
2.3.3 Applying the Model
We first apply the ResAttU-Net to derive the steady-state CBV maps in WT mice directly from their noncontrast pre-scans. In addition to demonstrating the ability to produce CBV in the normal brain tissue, we aimed to further verify its utility in the enhancement of pathology visibility and delineation of brain lesions as our second objective. For the first aim, 49 WT mice are used in the study, with a randomized 37-6-6 train-validation-test split. For the second aim, we randomly added 6 GBM mice scans to the training set and retrained the model. The performance of DeepContrast on tumor enhancement was tested on 4 GBM mice scans.
2.4 Evaluation and Statistical Analysis
To evaluate the performance of DeepContrast, we used a peak signal-to-noise ratio (PSNR) to assess the estimation error at the voxel level. Given the limitation of PSNR on capturing the perceptually relevant differences, we further calculated the structural similarity index (SSIM) to evaluate the accuracy of estimating a processed image on the structural level[PSNR]. In addition, we conducted Pearson and Spearman correlation analysis to assess the linear and monotonic associations between the CBV ground truth and DeepContrast respectively. For the GBM study, besides the voxel level comparison, we utilized the Dice similarity coefficient and Hausdorff distance to compare the performance of DeepContrast and the CBV ground truth in tumor segmentation.
3.1 Performance of DeepContrast in Normal Brain CBV Mapping
Example of the DeepContrast prediction and the quantitative metrics of the standalone 6 testing subjects are shown in Figure 3. DeepContrast captured the high contrast and fine details of small vessels with high similarity to the steady-state CBV ground truth in the normal brain tissue. Figure 3.a shows the CBV ground truth, predicted result, 20% low Gd dose CBV and noncontrast Pre image of the same mouse. The Pre image alone can provide promising prediction results that show strong enhancement compared to the 20% low-dose CBV that are consistent with the steady-state CBV maps. Figure 3.b shows the quantitative comparison of the DeepContrast and 20% Gd CBV map. DeepContrast clearly improved the CBV contrast derived from 20% Gd enhancement with a gain of 9.7% SSIM (p < 0.001), an increase of 5.3 dB in PSNR (p < 0.001), an improvement of 14.9% increase of P.R (Pearson Correlation, p < 0.05) and an 8.6% increase in S.R (Spearman Correlation, p < 0.05).
3.2 Performance of DeepContrast in GBM CBV Enhancement
To experiment and evaluate the utility of our DeepContrast method in the enhancement of pathology visibility and delineation of brain lesions, we retrained the same ResAttU-Net architecture with additional MRI scans of the GBM mouse model.
Figure 4.a-b show that the DeepContrast predicted results perform significantly better than the 20% Gd CBV in both visual assessment and the quantitative evaluation. Figure 4.a shows the predicted result of one slice of the same tumor subject. Compared with 20% Gd CBV, DeepContrast generated from the T2W Pre scan shows similar contrast level to the ground truth. The consistency can be observed in the fine structure and significant contrast enhancement in both normal tissue and the tumor region. Figure 4.b is the detailed quantitative metrics evaluation of 4 randomly selected testing tumor subjects. Compared with the 20% Gd CBV, DeepContrast has a 4.8% increase in SSIM (P < 0.05), a 2.0 dB increase in PSNR (P < 0.05), and a 19.6% increase in P.R (Pearson Correlation, P < 0.05) and a 2.6% increase in S.R (Spearman Correlation, P < 0.05). Figure 4 confirms that the DeepContrast predicted results perform significantly better than the 20% Gd CBV in both visual assessment and the quantitative evaluation.
Figure 4.c-d further confirms that the DeepContrast models with the input of Pre only image significantly outperformed the 20% Gd for the tumor region. Figure 4.c shows the 3D rendering results of CBV ground truth vs DeepContrast in the FCM segmented tumor region of a single tumor subject. Compared to low dose CBV, both the two DeepContrast models derived the tumor region more precisely with a similar contrast level to the ground truth.
Figure 4.d shows the quantitative comparison between the DeepContrast predictions and the 20% Gd CBV of the tumor region. The DeepContrast performed significantly better than the 20% Gd of the tumor region, with 47.0% Dice coefficient increases (P < 0.05) and 5.5 pixels Hausdorff distance reductions (P < 0.05).
4 Discussion and Conclusions
Findings of Gd retention necessitate efforts to develop novel approaches to MRI that can decrease or even eliminate Gd exposure. In the past, there have been several attempts to develop noncontrast MRI sequences such as ASL, TOF, and VASO [RN2]. While such sequences have been successful and applied clinically, Gd based imaging still offers an unparalleled level of information in comparison. For example, Gd is widely used for steady-state CBV fMRI imaging to map basal brain metabolism in both mice and humans [RN12, RN16]. CBV has been proven to be tightly coupled to regional metabolism in healthy and diseased brains [RN36, RN37] and is useful in studying cognitive aging [RN36], Alzheimer disease (AD) [RN37] and tumors [RN38]. Gd-enhanced CBV is also well suited to detect GBM-related regional hyperactive angiogenesis and blood-brain-barrier (BBB) leakage which has become a potential imaging biomarker for GBM detection and grading [RN23, RN24, RN25, RN26].
The question remains whether it is feasible to efficiently gather Gd-based information directly from noncontrast MRI scans to ensure patient safety and limit Gd exposure. Removing the Gd contrast entirely while retaining diagnostic information could have a large impact on both patient well-being and reduction of imaging time and costs. Recently, 3D Bayesian U-Net was applied to a dataset from patients with brain tumors and healthy subjects to predict contrast enhancement from a comprehensive multiparametric MRI protocol including T1w, T2w, T2w fluid-attenuated inversion recovery, diffusion-weighted imaging, and susceptibility-weighted imaging, all acquired without any Gd injections [Can]. This method was limited in its inability to predict presence of small vessels and an inevitably long scan time likely to exhaust patients due to utilization of 10 multiparametric MRI scans as its input. With these limitations in mind, DeepContrast was developed to rely solely on information extracted from the most commonly acquired structural MRI scans. Compared to the 3D Bayesian U-Net, our approach provides several improvements to predicting contrast enhancement. First, we demonstrate that Gd contrast in brain MRI can be directly derived from a single noncontrast T2W MRI in both normal brain tissue and brain lesion. High PSN, high SSIM and high correlations between the DeepContrast and the Gd-enhanced ground truth suggest that our proposed method does not cause significant quality degradation. High accuracy of GBM segmentations suggest DeepContrast is capable of detecting GBM with performance similar to full-dose Gd-enhanced scans. Furthermore, our deep learning method is based on a hybrid deep residual attention-aware network and is the first network to use an attention residual mechanism to process brain MRI scans. The basic architecture of the proposed network is a 2D U-Net that extracts contextual information combining low-level feature maps with high-level ones. Attention modules are stacked such that the attention-aware features change adaptively in the deeper layers of the network. This is made possible by residual learning. Finally, our method is capable of producing CBV mapping of small vessels with high fidelity.
While our method of DeepContrast enhancement shows promising results, there are several limitations that must be considered. Regarding application of the network to GBM mouse models, our training dataset lacking longitudinal scans to capture GBM contrast at different tumor stages. To improve the robustness and accuracy of DeepContrast for GBM enhancement, we could enrich our current training dataset by adding MRI scans of mice injected with GBM cells at various stages and different locations. Beyond diversifying the dataset, to deal with the intensity difference within and across subjects, recent advancements in intensity standardization / normalization would arguably benefit the estimation of the DeepContrast enhancement when translating our proposed model to human data [RN49]
. An additional factor and potential limitation to consider is loss function parameters. MSE was chosen as the cost function in our current model, but the training strategy could be further improved by adding other evaluation parameters to the loss function. Overall, future works will incorporate larger datasets, enhanced image pre-processing, and model advancement to optimize both the network and training pipeline.
Despite aforementioned limitations, our proposed method currently can be used to generate Gd contrast in brain MRI directly from T2W MRI scans with complete omission of GBCAs. DeepContrast is a promising technique with the potential to offer benefits to patient care and the healthcare system through reduction of Gd exposure, scan time, and cost. Of note, our work should be regarded as a proof-of-concept study in animal models; future human studies will be required to validate its clinical utility.
This study was funded by the Seed Grant Program and Technical Development Grant Program and Matheson Foundation (UR010590). This study was performed at the Zuckerman Mind Brain Behavior Institute MRI Platform, a shared resource.