High-Frequency Space Diffusion Models for Accelerated MRI

08/10/2022
by   Chentao Cao, et al.
NetEase, Inc
56

Denoising diffusion probabilistic models (DDPMs) have been shown to have superior performances in MRI reconstruction. From the perspective of continuous stochastic differential equations (SDEs), the reverse process of DDPM can be seen as maximizing the energy of the reconstructed MR image, leading to SDE sequence divergence. For this reason, a modified high-frequency DDPM model is proposed for MRI reconstruction. From its continuous SDE viewpoint, termed high-frequency space SDE (HFS-SDE), the energy concentrated low-frequency part of the MR image is no longer amplified, and the diffusion process focuses more on acquiring high-frequency prior information. It not only improves the stability of the diffusion model but also provides the possibility of better recovery of high-frequency details. Experiments on the publicly fastMRI dataset show that our proposed HFS-SDE outperforms the DDPM-driven VP-SDE, supervised deep learning methods and traditional parallel imaging methods in terms of stability and reconstruction accuracy.

READ FULL TEXT VIEW PDF

page 6

page 7

page 8

07/19/2021

Frequency-Supervised MR-to-CT Image Synthesis

This paper strives to generate a synthetic computed tomography (CT) imag...
03/29/2021

Estimation of ergodic square-root diffusion under high-frequency sampling

We study the Gaussian quasi-likelihood estimation of the parameter θ:=(α...
10/23/2019

Learning Priors in High-frequency Domain for Inverse Imaging Reconstruction

Ill-posed inverse problems in imaging remain an active research topic in...
03/27/2018

A Divide-and-Conquer Approach to Compressed Sensing MRI

Compressed sensing (CS) theory assures us that we can accurately reconst...
05/27/2018

Synergistic Reconstruction and Synthesis via Generative Adversarial Networks for Accelerated Multi-Contrast MRI

Multi-contrast MRI acquisitions of an anatomy enrich the magnitude of in...
12/26/2019

Multi-Site High-Frequency Stochastic Precipitation Generator Using Censored Skew-Symmetric Distributions

Due to improved measuring instruments, an accurate stochastic weather ge...

1 Introduction

Due to the limitations of the Magnetic Resonance Imaging (MRI), MRI scanning takes a long time, which limits the widespread use of MRI. The k-space data is usually highly under-sampled to reduce the time required for MR scanning. How to reconstruct high-quality images from highly under-sampled k-space data is a pivotal topic in MR reconstruction. In the past, fast MRI was performed mainly through parallel imaging and compressed sensing methods (Lingala et al. [2011], Lin and Fessler [2019], Majumdar [2015]). With the development of deep learning methods, deep learning MRI reconstruction methods have achieved superior reconstruction results (Aggarwal et al. [2018], Cui et al. [2021], Cao et al. [2022], Ke et al. [2021, 2020], Cheng et al. [2020], Huang et al. [2021]).

Recently, diffusion models have achieved excellent performance in many image processing tasks, including two types of generative models, score-based and DDPM (Song and Ermon [2019], Ho et al. [2020]), which provide new approaches to solve the inverse image reconstruction problem. In the framework of continuous SDE, Song et al. [2021] unified them and proposed three forms of SDEs, VE-SDE, VP-SDE and sub-VP-SDE, to further improve the performance of diffusion models. Currently, Jalal et al. [2021], Song et al. [2022], Chung and Ye [2022] have achieved impressive results in MRI reconstruction tasks using score-based generative model or VE-SDE.

The initial value of the reversal process of VP-SDE (corresponding to the reconstruction process of MRI) is gaussian white noise, which is not related to the original image. In MR reconstruction, the fully sampled original image is not available, while the initial value of the VE-SDE reverse process requires the original image, which is not available in reality. So VP-SDE is more suitable to be applied to MR reconstruction tasks than VE-SDE. The reverse-time SDE iteration rule of VP-SDE is:

(1)

where is considered to be the reconstructed MR image, is the trained score-based model, and is a given coefficient to control the noise level. From optimization perspective, the reverse VP-SDE can be viewed as the Stochastic gradient Langvien descent algorithm for the following problem:

(2)

where are positive weights. When , the objective function may obtain the minimum value at , which means that there is a risk of divergence of the VP-SDE.

1.1 Contributions

Motivated by the above mentioned problems and combined with MRI, we propose a more stable diffusion model. Specifically, the main contributions of our work are summarized as follows.

  • [leftmargin = *]

  • For MRI reconstruction, we propose a high-frequency SDE diffusion model, termed HFS-SDE, in which the energy of the SDE sequence does not expand infinitely as time evolves. Meanwhile, the diffusion process only takes place at high frequencies, which can more effectively characterize the image high-frequency prior.

  • Since the HFS-SDE diffuses only in high-frequency space, the initial value of its reverse process can be allowed to zero filling images instead of Gaussian white noise. From the optimization point of view, a good initial value facilitates a faster and more accurate algorithm convergence to the desired solution.

  • Experiments show that our HFS-SDE consistently outperforms its competitor VP-SDE and other supervised deep learning methods and conventional parallel imaging methods regarding MR reconstruction accuracy and stability.

1.2 Related Works

Several related works have used the diffusion model to accelerate MRI reconstruction due to its superior performance. Jalal et al. [2021] first applied the score-based generative model to fast MR reconstruction. In the framework of continuous SDE, Song et al. [2022] and Chung and Ye [2022] proposed to apply VE-SDE to the MR reconstruction. Unlike these three works, we propose a new SDE, which performs the diffusion process only in the high-frequency space and can better reconstruct the high-frequency details of MR images.

Xie and Li [2022] proposed measurement-conditioned DDPM (MC-DDPM) for accelerated MRI reconstruction. MC-DDPM trains the diffusion model by adding an under-sampled mask to the training data. However, the diffusion process of MC-DDPM is performed in the k-space domain, which leads to the case that MC-DDPM cannot handle data with different number of coils. Unlike MC-DDPM, the diffusion process of our method is performed in the high-frequency space of the image. The method we proposed samples only the central region of the k-space data. Moreover, our proposed method is based on the SDE continuous framework, and for the drawback of VP-SDE, we propose a new form of SDE called HFS-SDE.

The following sections of the paper are organized as follows: Section 2 introduces the backward of generative diffusion processes, Section 3 describes the proposed method, Section 4 provides the experimental results. Discussion and conclusion are given in Section 5 and Section 6.

2 Backward

Song et al. [2021] proposed a unified framework that generalizes two types of generative models (Song and Ermon [2019], Ho et al. [2020]) through the lens of stochastic differential equations (SDEs). The diffusion process is described as the solution of the following SDE:

and are called the drift and diffusion coefficients of x, respectively. is the standard Wiener process. The reverse diffusion process is obtained from the following reverse-time SDE:

is known as score function and can be generated by sampling from . The score network estimates the score function, and the loss function of the network is

(3)

is a positive weighting function, is uniformly sampled over .

In this unified framework, DDPM is expanded into the form of VP-SDE:

The perturbation kernel of the VP-SDE can be derived from Särkkä and Solin [2019]:

The score model can be trained by Eq. (3) and finally, the image is generated by the reverse iteration rule of Eq. (1).

3 Method: High-Frequency Space Diffusion Stochastic Differential Equations

Since the energy of the image is mainly concentrated in the low frequency region, we solve the instability problem of VP-SDE by constructing the diffusion process in the high-frequency region of the image. We define and as high-frequency and low-frequency operators. In particular, in the multi-channel scenario, , , indicates the number of coils, and denotes the sensitivity of the th coil. is the Fourier operator, is the operator to obtain the central low-frequency data that only takes center region of k-space and

denotes identity matrix.

The discrete Markov chain form of the forward HFS-SDE is as follows

(4)

are given coefficients to control the noise level and . By introducing the auxiliary variable , Eq. (4) can be rewritten as

When , can be written as and . Let ,

(5)

Eq. (5) converges to

By Särkkä and Solin [2019] and let for , the perturbation kernel of the discrete form of the HFS-SDE can be derived as

The high-frequency operator is exponential power, which is nonlinear and impractical to implement directly. Assuming a coefficient of before , we solve this problem with the following trick

Since , we can get

The mean and covariance matrix of the perturbation kernel can be re-expressed as

The parameters in the network is trained via the following loss function:

is the positive weighting function. The network structure in HFS-SDE is DDPM++, which is improved by Song et al. [2021]. For the MRI inverse problem, we can only acquire the under-sampled k-space measurements y, so we aim to reconstruct the MR image based on . Based on the trained network, we can perform the following reverse HFS-SDE conditional on to obtain the reconstructed image. (Anderson [1982])

Further, we have

Hence, The reconstruction process is shown in Alg. 1.

1:. , is the under-sampling mask
2:
3:for  to  do
4:     
5:     
6:      Controllable generation.
7:     
8:     
9:     for  to  do
10:         
11:         
12:          Controllable generation.
13:         
14:         
15:         
16:     end for
17:     
18:end for
19:return
Algorithm 1 PC Sampling (HFS-SDE).

4 Experiments

4.1 Experimental Details

We trained on the multichannel fastMRI T1-weighted dataset (Zbontar et al. [2018], Knoll et al. [2020]) using 34 individuals with a total of 1002 images, and we cropped the image size to 320x320 and tested on T1-weighted knee scans and T2-weighted brain scans.

We compared with the traditional parallel imaging method, supervised deep learning method and DDPM-driven VP-SDE (Pruessmann et al. [1999], Zhang and Ghanem [2018], Song et al. [2021]

). Each method was tuned to its best achievable result, and the Vp-SDE and HFS-SDE were trained with 500 epochs. The parameters required in Alg. 

1 are set as follows: .

4.2 Experimental Results

The reconstruction results of the multi-channel uniformly undersampled by 8-fold with 18 ACS lines are shown in Fig. 1. The reconstruction results of HFS-SDE are remarkably better than these three methods, and the high-frequency details of the image are accurately reconstructed.

Figure 1: The reconstruction results in the multi-channel, 8-fold under-sampling scene. The first row shows the reconstruction results of SENSE, ISTA-Net, VP-SDE and HFS-SDE. The yellow numbers at the bottom of the first row indicate PSNR (dB) and SSIM. The second row shows the enlarged view of the ROI region (indicated by the yellow box in the first row), and the third row shows the error map of the ROI region. In the training process, operator in HFS-SDE captures 22 central lines in k-space. The under-sampling mask is shown in the lower left corner, which is also used for ISTA-Net training.

Fig. 2 shows the reconstruction results in the extreme case of 10 under-sampled. HFS-SDE still shows no artifacts and reconstructs the image details well. Because the VP-SDE takes the optimal solution when the image energy is maximized, the VP-SDE is severely affected by artifacts in high acceleration scenes. The HFS-SDE can suppress the artifacts well, which indicates that our method can well correct the unstable defects of VP-SDE.

Figure 2: The reconstruction results in the multi-channel, 10-fold under-sampled scene. The first row shows the reconstruction results of SENSE, ISTA-Net, VP-SDE and HFS-SDE. The yellow numbers at the bottom of the first row indicate PSNR (dB) and SSIM. The second row shows the enlarged view of the ROI region (indicated by the yellow box in the first row), and the third row shows the error map of the ROI region. In the training process, operator in HFS-SDE captures 24 central lines in k-space. The under-sampling mask is shown in the lower left corner, which is also used for ISTA-Net training.

We conducted anatomy-shift and mask shift experiments to demonstrate the generalization performance of the HFS-SDE. In the anatomy-shift experiment, ISTA-Net, VP-SDE and HFS-SDE were trained on T1-weight knee data and tested on T2-weight brain data, and the reconstruction results are shown in Fig. 3. In the mask-shift experiment, the under-sampling mask in the test phase was the cartesian mask. In the training process, the mask imposed in the ISTA-Net was a uniform 6-fold mask. operator in HFS-SDE captures 22 central lines in k-space, and the reconstruction results are shown in Fig. 4. When the distribution of training and test data is significantly different, the performance of the supervised training method (ISTA-Net) is greatly degraded, while the diffusion models method can still reconstruct the image accurately. In particular, the HFS-SDE significantly outperforms the VP-SDE in recovering high-frequency details.

Figure 3: Anatomy-shift results. The reconstruction results in the multi-channel, 8-fold under-sampling scene. The first row shows the reconstruction results of SENSE, ISTA-Net, VP-SDE and HFS-SDE. The yellow numbers at the bottom of the first row indicate PSNR (dB) and SSIM. The third row shows the error map of the ROI region. In the training process, operator in HFS-SDE captures 22 central lines in k-space. The under-sampling mask is shown in the lower left corner, which is also used for ISTA-Net training.
Figure 4: Mask-shift results. The reconstruction results in the multi-channel, 6-fold under-sampling scene. The first row shows the reconstruction results of SENSE, ISTA-Net, VP-SDE and HFS-SDE. The yellow numbers at the bottom of the first row indicate PSNR (dB) and SSIM. The second row shows the enlarged view of the ROI region (indicated by the yellow box in the first row), and the third row shows the error map of the ROI region. In the training process, operator in HFS-SDE captures 22 central lines in k-space and ISTA-Net used the uniform 8-fold under-sampling mask in Fig. 1. The random uniform under-sampling mask used in the test process is shown in the lower left corner.

5 Discussion

Because the diffusion process of HFS-SDE is performed only in high-frequency space, the initial value of the reverse diffusion is the low-frequency part of the image plus the Gaussian noise in the high-frequency part. When the operator selects a less low-frequency region, it will easily lead to excessive energy instability. However, when the selected low-frequency region is too large (greatly exceeding the number of ACS lines of the under-sampling mask), the reconstruction results will have lost high-frequency details. Hence, the range selection of the operator has a large impact on the HFS-SDE performance. We conducted experiments in under-sampling scenarios of 10, and the ACS lines number of the under-sampling mask was chosen to be 18. The experimental results are detailed in Fig. 5. When the under-sampling rate is low (less than 6), the low-frequency region selected by the operator should make consistent with the number of ACS lines in the centre of the under-sampling mask. As the under-sampling rate increases, the low-frequency region of the operator should be gradually increased to suppress the generation of artifacts.

Figure 5: Range selection for the central low frequency region in HFS-SDE. The reconstruction results in the multi-channel, 10-fold under-sampling scene. The first row shows the reconstruction results and the yellow numbers at the bottom of the first row indicate PSNR (dB) and SSIM. The third row shows the error map of the ROI region. In the training process, operator in HFS-SDE captures 18, 20, 22, 24 central lines in k-space.

6 Conclusions

In this paper, we propose a new SDE for fast MR reconstruction from an optimization point of view, whose diffusion process is in the high-frequency part of the image. The energy concentrated low-frequency part of the MR image is no longer amplified, and the diffusion process focuses more on acquiring high-frequency prior information. It not only improves the stability of the diffusion model but also enables more accurate recovery of high-frequency details in MR images. We conducted experiments on the multichannel fastMRI dataset, and the experiments show that our method outperforms VP-SDE, supervised deep learning methods and traditional parallel imaging methods in terms of stability and reconstruction accuracy.

References

  • H. K. Aggarwal, M. P. Mani, and M. Jacob (2018) MoDL: model-based deep learning architecture for inverse problems. IEEE Transactions on Medical Imaging 38 (2), pp. 394–405. Cited by: §1.
  • B. D. Anderson (1982) Reverse-time diffusion equation models. Stochastic Processes and their Applications 12 (3), pp. 313–326. Cited by: §3.
  • C. Cao, Z. Cui, Q. Zhu, D. Liang, and Y. Zhu (2022) PS-net: deep partially separable modelling for dynamic magnetic resonance imaging. arXiv preprint arXiv:2205.04073. Cited by: §1.
  • J. Cheng, W. Huang, Z. Cui, Z. Ke, L. Ying, H. Wang, Y. Zhu, and D. Liang (2020) Learning data consistency for mr dynamic imaging. Learning. Cited by: §1.
  • H. Chung and J. C. Ye (2022) Score-based diffusion models for accelerated mri. Medical Image Analysis, pp. 102479. Cited by: §1.2, §1.
  • Z. Cui, J. Cheng, Q. Zhu, Y. Liu, S. Jia, K. Zhao, Z. Ke, W. Huang, H. Wang, Y. Zhu, et al. (2021) Equilibrated zeroth-order unrolled deep networks for accelerated mri. arXiv preprint arXiv:2112.09891. Cited by: §1.
  • J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems 33, pp. 6840–6851. Cited by: §1, §2.
  • W. Huang, Z. Ke, Z. Cui, J. Cheng, Z. Qiu, S. Jia, L. Ying, Y. Zhu, and D. Liang (2021) Deep low-rank plus sparse network for dynamic mr imaging. Medical Image Analysis 73, pp. 102190. Cited by: §1.
  • A. Jalal, M. Arvinte, G. Daras, E. Price, A. G. Dimakis, and J. Tamir (2021) Robust compressed sensing mri with deep generative priors. Advances in Neural Information Processing Systems 34, pp. 14938–14954. Cited by: §1.2, §1.
  • Z. Ke, Z. Cui, W. Huang, J. Cheng, S. Jia, L. Ying, Y. Zhu, and D. Liang (2021) Deep manifold learning for dynamic mr imaging. IEEE Transactions on Computational Imaging 7, pp. 1314–1327. Cited by: §1.
  • Z. Ke, W. Huang, J. Cheng, Z. Cui, S. Jia, H. Wang, X. Liu, H. Zheng, L. Ying, Y. Zhu, et al. (2020) Deep low-rank prior in dynamic mr imaging. arXiv preprint arXiv:2006.12090. Cited by: §1.
  • F. Knoll, J. Zbontar, A. Sriram, M. J. Muckley, M. Bruno, A. Defazio, M. Parente, K. J. Geras, J. Katsnelson, H. Chandarana, et al. (2020)

    FastMRI: a publicly available raw k-space and dicom dataset of knee images for accelerated mr image reconstruction using machine learning

    .

    Radiology. Artificial intelligence

    2 (1).
    Cited by: §4.1.
  • C. Y. Lin and J. A. Fessler (2019) Efficient dynamic parallel mri reconstruction for the low-rank plus sparse model. IEEE Transactions on Computational Imaging 5 (1), pp. 17–26. External Links: Document Cited by: §1.
  • S. G. Lingala, Y. Hu, E. DiBella, and M. Jacob (2011) Accelerated dynamic mri exploiting sparsity and low-rank structure: k-t slr. IEEE Transactions on Medical Imaging 30 (5), pp. 1042–1054. External Links: Document Cited by: §1.
  • A. Majumdar (2015) Improving synthesis and analysis prior blind compressed sensing with low-rank constraints for dynamic mri reconstruction. Magnetic Resonance Imaging 33 (1), pp. 174–179. Cited by: §1.
  • K. P. Pruessmann, M. Weiger, M. B. Scheidegger, and P. Boesiger (1999) SENSE: sensitivity encoding for fast mri. Magnetic Resonance in Medicine: An Official Journal of the International Society for Magnetic Resonance in Medicine 42 (5), pp. 952–962. Cited by: §4.1.
  • S. Särkkä and A. Solin (2019) Applied stochastic differential equations. Vol. 10, Cambridge University Press. Cited by: §2, §3.
  • Y. Song and S. Ermon (2019) Generative modeling by estimating gradients of the data distribution. Advances in Neural Information Processing Systems 32. Cited by: §1, §2.
  • Y. Song, L. Shen, L. Xing, and S. Ermon (2022) Solving inverse problems in medical imaging with score-based generative models. In International Conference on Learning Representations, External Links: Link Cited by: §1.2, §1.
  • Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021) Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, External Links: Link Cited by: §1, §2, §3, §4.1.
  • Y. Xie and Q. Li (2022) Measurement-conditioned denoising diffusion probabilistic model for under-sampled medical image reconstruction. arXiv preprint arXiv:2203.03623. Cited by: §1.2.
  • J. Zbontar, F. Knoll, A. Sriram, T. Murrell, Z. Huang, M. J. Muckley, A. Defazio, R. Stern, P. Johnson, M. Bruno, et al. (2018) FastMRI: an open dataset and benchmarks for accelerated mri. arXiv preprint arXiv:1811.08839. Cited by: §4.1.
  • J. Zhang and B. Ghanem (2018) ISTA-net: interpretable optimization-inspired deep network for image compressive sensing. In

    Proceedings of the IEEE conference on computer vision and pattern recognition

    ,
    pp. 1828–1837. Cited by: §4.1.