1 Introduction
Magnetic resonance fingerprinting (MRF) [12] is a new quantitative imaging paradigm that allows fast and parallel measurement of multiple tissue properties in a single acquisition, unlike conventional methods that quantify one specific tissue property at a time. MRF randomizes multiple acquisition parameters to generate unique signal evolutions, called “fingerprints”, that encode information of multiple tissue properties of interest. time points are usually acquired and one image is reconstructed for each time point. Dictionary matching (DM) is then used to match the fingerprint at each pixel to a predefined dictionary of fingerprints associated with a wide range of tissue properties.
To improve the clinical feasibility of MRF, many studies have investigated replacing DM with deep neural networks to accelerate tissue mapping
[2, 4, 8, 9, 3]. However, these methods, similar to DM, operate on the reconstructed MRF images and are therefore still limited by the speed and computational efficiency of conventional reconstruction methods. Particularly, since MRF employs a spiral space sampling trajectory for robustness to motion [12], the reconstruction is nontrivial and more timeconsuming than the Cartesian case.A major challenge is that the computationally efficient inverse fast Fourier transform (FFT) cannot be directly applied to nonCartesian data. Besides, the density of the samples varies along the nonCartesian trajectory and must be compensated for to ensure highquality reconstruction. Most existing nonCartesian MRI reconstruction methods thus consist of independent steps that are not optimized endtoend, relying heavily on NonUniform Fast Fourier Transform (NUFFT) [5].
Fewer deep learning based reconstruction methods focus on nonCartesian sampling [6, 17] than on Cartesian sampling [21, 18, 19, 16]. AUTOMAP [22] attempts to use a fullyconnected network (FNN) to learn the full mapping from raw space data to images, including the Fourier transform (FT). Although FNN makes no assumptions on the sampling pattern and aligns with the nature of FT, the network size is quadratic with the image size (
), incurring immense memory costs that limit scalability to large images, especially to highdimensional MRF data with thousands of time frames. Moreover, MRF involves an additional tissue mapping that may require a tailored network architecture based on convolutional neural networks (CNNs)
[3, 4]or recurrent neural network (RNNs)
[9] for optimal performance.Our aim in this paper is to introduce a framework for realtime tissue quantification directly from nonCartesian space MRF data using only regular 2D convolutions and FFT, providing a computationally more feasible solution to highdimensional MR data reconstruction and allowing greater flexibility in network design. Mimicking DFT directly with locallyconnected CNNs is not effective since every point in the space has a global effect on the image. Our approach is inspired by the gridding process in NUFFT [1, 5]. However, instead of explicitly incorporating the memorycostly gridding kernel of NUFFT as in [6, 17], we show for the first time that gridding and tissue mapping can be performed seamlessly in a single mapping. Experiments on 2D and 3D MRF data demonstrate that our completely endtoend framework achieves results on par with stateoftheart methods that use more complicated reconstruction schemes while being orders of magnitude faster. To the best of our knowledge, no prior methods have demonstrated the feasibility of endtoend nonCartesian MRI reconstruction for data as highdimensional as MRF in a single framework dealing with both reconstruction and tissue mapping simultaneously without the need for NUFFT.
2 Methods
2.1 Problem Formulation
With the MRF sequence employed in this study, only th of the full data, i.e., a single spiral, is collected for each time frame for significant acceleration. The original MRF framework first reconstructs an image from each spiral of length using NUFFT, leading to highlyaliased images. The image series are then mapped to the corresponding tissue property T1 and T2 maps with image dimensions .
In contrast, our approach directly maps the highlyundersampled spiral space MRF data to the Cartesian space of the or map, and finally to the image space of or map simply via inverse FFT (Fig. 1).
Let each data point in
space be represented as a location vector
and a signal value . To grid the signal , convolution is applied via weighted summation of the signal contributions of neighboring sampled data points of :(1) 
where denotes the gridding kernel centered at , and is the density compensation factor for data point . Points in sparsely sampled regions are associated with greater compensation factor. Density compensation is required because in nonCartesian imaging, the central space (lowfrequency components) is usually more densely sampled than the outer space (highfrequency components).
2.2 Proposed Framework
Instead of performing gridding in space and tissue mapping in image space separately as in most existing methods [2, 4], we propose to perform tissue mapping directly from space
. This allows gridding and tissue mapping for thousands of time frames to be performed simultaneously via a single CNN, which is key to achieving realtime tissue quantification. Applying CNNs to nonCartesian data, however, is not straightforward. Here, without actually interpolating each grid point on the Cartesian
space of MRF frames, our key idea is to directly use the signal time courses of nearest neighboring spiral points of a target grid pointto infer the corresponding tissue properties (Fig. 2(a)), based on their relative positions to the target grid point and their densities (Fig. 2(b)). K NearestNeighbor (KNN) search and density estimation only need to be computed once for each trajectory and prestored; therefore the required time cost is negligible. Individual components of the proposed framework are described next.
2.3 SlidingWindow Stacking of Spirals
In MRF, data are typically sampled incoherently in the temporal dimension via a series of rotated spirals. Each MRF time frame is highly undersampled with one spiral. Here, we combine every 48 temporally consecutive spirals in a slidingwindow fashion for full space coverage (Fig. 2(a, Left)). This reduces the number of time frames from to and allows each spiral point to be associated with a dimensional feature vector . The input to the network thus become: . From Eq. (1), sampled points are gridded based only on their relative positions to the target grid point, i.e., . Thus, as exemplified in Fig. 2(a, Right), different contributes differently according to its spatial proximity with respect to the center grid point in a point local neighborhood.
2.4 Learned Density Compensation
In nonCartesian imaging, measurement density varies in space and is typically dense at the center and sparse at the peripheral of space. Density compensation (DC) can thus be viewed as a function of data location on a space sampling trajectory with respect to the space center. This is different from gridding using local weighting with respect to a target grid point. Thus, we propose to parameterize the DC function using 2D polar coordinates of the sampled points:
(2) 
where and . Straightforward choices of are and . However, rather than fixing and handcrafting , we learn the DC function to adapt to different sampling trajectories via a network that is sensitive to sample locations. This is achieved by directly concatenating the polar coordinates with the spiral point features, inspired by “CoordConv” [10]. By simply giving the convolution access to the input coordinates, the network can adaptively decide where to compensate by learning different weights for the density features associated with different spiral points. This is unlike conventional methods where DC weighting functions are computed analytically [7] or iteratively [14]. See [10] for more information on translationvariant CoordConv.
2.5 Tissue Mapping via Agglomerated Neighboring Features
The features for each target grid point are agglomerated from its nearest neighbors from a stack of spirals. This transforms the spiral data to a grid, allowing regular 2D convolutions to be applied directly. Concatenating point features with additional dimensional point distribution information required by gridding and density compensation leads to input . Since our framework does not emphasize on and is not limited to a certain network architecture, we extend an existing UNet [15] based MRF tissue quantification network [4] to make it fully endtoend, mapping the agglomerated features directly to the corresponding tissue property maps. To improve computational efficiency, a micronetwork is employed preceding the quantification network to reduce the dimensionality of each target grid feature vector
by a shared linear transformation
, implemented as an convolution:(3) 
where denotes concatenation, , and . The resulting feature map is then fed to the quantification network.
2.5.1 Network Parameters and Training.
Our network backbone consists of a micronetwork and a 2D UNet, which is lighter than AUTOMAP [22]. AUTOMAP is computationally expensive when applied to MRF ( params). The micronetwork is composed of four
convolutional layers, each followed by batch normalization and ReLU. The number of output channels of all
convolutions is ( for T1, and for T2, chosen by cross validation). The input channel number of the micronetwork is , where . The network was trained in batches of 2 samples and optimized via ADAM with an initial learning rate of , which was decayed byafter each epoch. Following
[4], relativeL1 was used as the objective function. Two GPUs (TITAN X, G) were used for training.3 Experiments and Results
3.0.1 Datasets.
2D MRF datasets were acquired from six normal subjects, each consisting of to scans. For each scan, a total of MRF time points were acquired and each contains only one spiral readout of length . Two 3D MRF datasets were used for evaluation. The first 3D MRF dataset with a spatial resolution of mm were collected from three subjects, each covering slices. A total of time points were acquired for each scan. The second 3D MRF datasets were acquired from six volunteers with a high isotropic resolution of mm, each covering slices. time points were collected for each scan. For both 3D datasets, FFT was first applied in the sliceencoding direction, and then the data of each subject were processed slicebyslice, just as in the 2D case. All MRI measurements were performed on a Siemens 3T scanner with a channel head coil. Real and imaginary parts of the complexvalued MRF signals are concatenated. For acceleration, only the first time frames in each 2D MRF scan and the first in each 3D MRF scan were used for training. The training data size for each 2D and 3D scan is and (or ) (# coils # spiral readouts # time frames), respectively. The groundtruth and maps with voxels were obtained via dictionary matching using all time frames in 2D MRF and all (or ) time frames in 3D MRF.
3.0.2 Experimental Setup.
1) We compared our endtoend approach with four stateoftheart MRF methods: a UNet based deep learning method (SCQ) [4], dictionary matching (DM) [12], SVDcompressed dictionary matching (SDM) [13], and a lowrank approximation method (LowRank) [11]. Note that these competing methods require first reconstructing the image for each time frame using NUFFT [5]. Leaveoneout cross validation was employed. 2) We also compared our adaptive gridding with typical handcrafted gridding methods, and investigated the effects of including the relative positions and density features. 3) As a proof of concept, we applied our method on the additional highresolution 3D MRF dataset for qualitative evaluation.
Method  MAE  SSIM  NRMSE  Recon. (s)  Patt. Match. (s)  Total (s)  

T1  T2  T1  T2  T1  T2  
DM  2.42  10.06  0.994  0.954  0.0150  0.0421  467  25  492 
SDM  2.42  10.05  0.994  0.954  0.0150  0.0421  467  10  477 
LowRank  2.87  8.17  0.991  0.960  0.0156  0.0302  3133  25  3158 
SCQ  4.87  7.53  0.992  0.968  0.0217  0.0309  9.73  0.12  9.85 
Ours  4.24  7.09  0.986  0.972  0.0258  0.0335  –  –  0.41 
3.0.3 Results and Discussion.
As shown in Table 1 and Table 2, our method performs overall best in T2 quantification accuracy and achieves competitive accuracy in T1 quantification with processing speed 24 times faster than a CNN method and 1,100 to 7,700 times faster than DM methods. Particularly, for 3D MRF, our method performs best for most metrics. Qualitative results are shown in Fig. 3 and Fig. 4. The higher T1 than T2 quantification accuracy is consistent with previous findings [20, 4]. Due to the sequence used in this study, the early portion of the MRF time frames, which were used for training, contain more information on T1 than T2. Hence, all methods are more accurate in T1 quantification. DM methods exhibit significant artifacts in T2 as indicated by the arrows in Fig. 3. Representative results for the additional highresolution 3D MRF data are shown in Fig. 5. In the ablation study shown in Table 3, our adaptive gridding performs better than typical handcrafted gridding methods.
Method  MAE  SSIM  NRMSE  Recon. (s)  Patt. Match. (s)  Total (s)  

T1  T2  T1  T2  T1  T2  
DM  5.89  12.19  0.996  0.968  0.0415  0.0521  140.56  17.01  157.57 
SCQ  16.58  16.74  0.933  0.919  0.0652  0.0479  8.58  0.11  8.69 
Ours  9.14  11.78  0.980  0.968  0.0389  0.0323  –  –  0.33 

Average  Bilinear  Gaussian  Ours  

No xy/density  xy  density  xy+density  

T1  5.59  5.27  5.53  5.24  4.34  4.48  4.24  
T2  7.74  8.48  7.95  9.05  8.43  7.37  7.09 
4 Conclusion
In this paper, we introduced a novel and scalable endtoend framework for direct tissue quantification from nonCartesian MRF data in milliseconds. With s per slice, slices for wholebrain coverage can be processed in one minute, allowing timely rescan decisions to be made in clinical settings without having to reschedule additional patient visits. It should be noted that the UNet based network backbone can be replaced with a more advanced architecture to further boost quantification accuracy. Our framework is also agnostic to the data sampling pattern, and thus can be potentially adapted to facilitate other nonCartesian MRI reconstruction tasks. We believe that our work will improve the clinical feasibility of MRF, and spur the development of fast, accurate and robust reconstruction techniques for nonCartesian MRI.
References
 [1] (2004) Handbook of mri pulse sequences. Elsevier. Cited by: §1.
 [2] (2018) MR fingerprinting deep reconstruction network (drone). Magnetic resonance in medicine 80 (3), pp. 885–894. Cited by: §1, §2.2.
 [3] (2020) Submillimeter mr fingerprinting using deep learning–based tissue quantification. Magnetic Resonance in Medicine 84 (2), pp. 579–591. Cited by: §1, §1.
 [4] (2019) Deep learning for fast and spatially constrained tissue quantification from highly accelerated data in magnetic resonance fingerprinting. IEEE transactions on medical imaging 38 (10), pp. 2364–2374. Cited by: §1, §1, §2.2, §2.5.1, §2.5, §3.0.2, §3.0.3.
 [5] (2003) Nonuniform fast fourier transforms using minmax interpolation. IEEE transactions on signal processing 51 (2), pp. 560–574. Cited by: §1, §1, §3.0.2.
 [6] (2019) Space deep learning for accelerated mri. IEEE transactions on medical imaging 39 (2), pp. 377–386. Cited by: §1, §1.
 [7] (1997) Density compensation functions for spiral mri. Magnetic Resonance in Medicine 38 (1), pp. 117–128. Cited by: §2.4.
 [8] (2018) Deep learning for magnetic resonance fingerprinting: accelerating the reconstruction of quantitative relaxation maps. In Proceedings of the 26th Annual Meeting of ISMRM, Paris, France, Cited by: §1.

[9]
(2019)
RinQ fingerprinting: recurrenceinformed quantile networks for magnetic resonance fingerprinting
. In International Conference on Medical Image Computing and ComputerAssisted Intervention, pp. 92–100. Cited by: §1, §1.  [10] (2018) An intriguing failing of convolutional neural networks and the coordconv solution. arXiv preprint arXiv:1807.03247. Cited by: §2.4.
 [11] (2017) Applications of low rank modeling to fast 3d mrf. In Proc Intl Soc Mag Reson Med, Vol. 25, pp. 129. Cited by: §3.0.2.
 [12] (2013) Magnetic resonance fingerprinting. Nature 495 (7440), pp. 187–192. Cited by: §1, §1, §3.0.2.
 [13] (2014) SVD compression for magnetic resonance fingerprinting in the time domain. IEEE transactions on medical imaging 33 (12), pp. 2311–2322. Cited by: §3.0.2.
 [14] (1999) Sampling density compensation in mri: rationale and an iterative numerical solution. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 41 (1), pp. 179–186. Cited by: §2.4.
 [15] (2015) Unet: convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computerassisted intervention, pp. 234–241. Cited by: §2.5.
 [16] (2017) A deep cascade of convolutional neural networks for dynamic mr image reconstruction. IEEE transactions on Medical Imaging 37 (2), pp. 491–503. Cited by: §1.
 [17] (2019) Nonuniform variational network: deep learning for accelerated nonuniform mr image reconstruction. In International Conference on Medical Image Computing and ComputerAssisted Intervention, pp. 57–64. Cited by: §1, §1.

[18]
(2020)
GrappaNet: combining parallel imaging with deep learning for multicoil mri reconstruction.
In
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
, pp. 14315–14322. Cited by: §1.  [19] (2019) Reducing uncertainty in undersampled mri reconstruction with active acquisition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2049–2058. Cited by: §1.
 [20] (2016) Maximum likelihood reconstruction for magnetic resonance fingerprinting. IEEE transactions on medical imaging 35 (8), pp. 1812–1823. Cited by: §3.0.3.
 [21] (2020) DuDoRNet: learning a dualdomain recurrent network for fast mri reconstruction with deep t1 prior. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4273–4282. Cited by: §1.
 [22] (2018) Image reconstruction by domaintransform manifold learning. Nature 555 (7697), pp. 487–492. Cited by: §1, §2.5.1.
Comments
There are no comments yet.