CG-SENSE revisited: Results from the first ISMRM reproducibility challenge

by   Oliver Maier, et al.

Purpose: The aim of this work is to shed light on the issue of reproducibility in MR image reconstruction in the context of a challenge. Participants had to recreate the results of "Advances in sensitivity encoding with arbitrary k-space trajectories" by Pruessmann et al. Methods: The task of the challenge was to reconstruct radially acquired multi-coil k-space data (brain/heart) following the method in the original paper, reproducing its key figures. Results were compared to consolidated reference implementations created after the challenge, accounting for the two most common programming languages used in the submissions (Matlab/Python). Results: Visually, differences between submissions were small. Pixel-wise differences originated from image orientation, assumed field-of-view or resolution. The reference implementations were in good agreement, both visually and in terms of image similarity metrics. Discussion and Conclusion: While the description level of the published algorithm enabled participants to reproduce CG-SENSE in general, details of the implementation varied, e.g., density compensation or Tikhonov regularization. Implicit assumptions about the data lead to further differences, emphasizing the importance of sufficient meta-data accompanying open data sets. Defining reproducibility quantitatively turned out to be non-trivial for this image reconstruction challenge, in the absence of ground-truth results. Typical similarity measures like NMSE of SSIM were misled by image intensity scaling and outlier pixels. Thus, to facilitate reproducibility, researchers are encouraged to publish code and data alongside the original paper. Future methodological papers on MR image reconstruction might benefit from the consolidated reference implementations of CG-SENSE presented here, as a benchmark for methods comparison.


page 22

page 23

page 24

page 25

page 26

page 27

page 28

page 29


Advancing machine learning for MR image reconstruction with an open competition: Overview of the 2019 fastMRI challenge

Purpose: To advance research in the field of machine learning for MR ima...

Validation and Generalizability of Self-Supervised Image Reconstruction Methods for Undersampled MRI

Purpose: To investigate aspects of the validation of self-supervised alg...

Improving reproducibility in synchrotron tomography using implementation-adapted filters

For reconstructing large tomographic datasets fast, filtered backproject...

Σ-net: Systematic Evaluation of Iterative Deep Neural Networks for Fast Parallel MR Image Reconstruction

Purpose: To systematically investigate the influence of various data con...

Deep Learning Guided Undersampling Mask Design for MR Image Reconstruction

In this paper, we propose a cross-domain networks that can achieve under...

4D Wavelet-Based Regularization for Parallel MRI Reconstruction: Impact on Subject and Group-Levels Statistical Sensitivity in fMRI

Parallel MRI is a fast imaging technique that enables the acquisition of...

Σ-net: Ensembled Iterative Deep Neural Networks for Accelerated Parallel MR Image Reconstruction

We explore an ensembled Σ-net for fast parallel MR imaging, including pa...