1 Introduction
The reconstruction of an unknown image from a set of noisy measurements is crucial in imaging inverse problems such as tomography [1, 2], phase retrieval [3, 4], and compressive sensing [5, 6]. When solving such illposed inverse problems, a widely used approach is to regularize the solution by using an image prior, such as the Tikhonov regularization [7] or total variation regularization (TV) [8].
Traditional image reconstruction methods assume accurate knowledge of the measurement operator characterizing the imaging system. In some cases, however, this operator is not known to sufficient accuracy, and can be modelled as depending on some parameters that have to be calibrated to obtain an accurate characterization of the imaging system. For example, calibration is essential in computerized tomography (CT) [9] when there is uncertainty in the projection angles used in data collection. In such settings, it is common to either manually calibrate the imaging system by using a known phantom [10, 11, 12, 13] or embed a calibration step into the reconstruction method [14, 15, 16].
There has been considerable recent interest in plugandplay priors (PnP) [17] and regularization by denoising (RED) [18], as frameworks for exploiting image denoisers as priors for image reconstruction. These methods achieve stateoftheart performance in a variety of imaging applications by integrating advanced denoisers that are not necessarily expressible in the form of a regularization function [19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29]. However, current PnP/RED algorithms assume exact knowledge of the measurement operator, limiting their ability to solve problems involving parametric uncertainties in the data acquisition.
We propose a new calibrated RED (CalRED) method that integrates an automatic parameter calibration procedure within the RED reconstruction. Each iteration of CalRED performs a update step, followed by a traditional update step, leading to the joint recovery of the unknown image and parameters . One of the key benefits of CalRED is that it can leverage powerful deep denoisers, such as DnCNN [30], for regularizing the inverse problem. We validate CalRED on CT by showing its ability to: (a) reduce the root mean squared error (RMSE) of the projection angles from to ; and (b) improve the imaging signal to noise ratio (SNR) from 10.16 dB (uncalibrated) to 21.13 dB (calibrated), with only a 0.95 dB gap relative to the oracle RED that knows the true angles. Note that while our focus in this letter is on RED, our calibration strategy is also fully compatible with PnP.
2 Background
2.1 Inverse Problems in Imaging
Consider the imaging problem specified by the linear model
(1) 
where denotes the uknown image, denotes the measurements, is the measurement operator characterizing the response of the imaging system, denotes the unknown parameters of the measurement operator, and is the noise. When the true value of the parameters is known, image reconstruction can be formulated as an optimization problem
(2) 
where is the datafidelity term that uses the measurement operator to ensure the consistency with the measurements and is the regularizer that imposes prior knowledge onto . For example, consider the smooth norm datafidelity term
(3) 
and the nonsmooth TV regularizer , where is the regularization parameter and is the image gradient [8]. Alternating direction method of multipliers (ADMM) [31] and fast iterative shrinkage/thresholding algorithm (FISTA) [32] are two common optimization algorithms that address the nonsmoothness of in image reconstruction via the proximal operator
(4) 
which can be interpreted as a
maximum a prior probability (MAP)
estimator for AWGN denoising.Deep learning has recently gained popularity for solving imaging inverse problems. An extensive review of deep learning in this context can be found in [33, 34, 35]. Instead of explicitly defining an optimization problem, the traditional deeplearning approach is based on training an existing network architecture, such as UNet [36], to invert the measurement operator by exploiting the natural redundancies in the imaging data [37, 38]
. It is common to first bring the measurements to the image domain and train the network to map the corresponding lowquality images to their clean target versions via supervised learning
[37, 38].2.2 Regularization by Denoising
RED is a recently introduced image recovery framework [18, 29] that seeks an images that satisfy
(5) 
where denotes the gradient of the datafidelity term using the true calibration parameters , is the regularization parameter, is a denoiser for the input noise level . RED thus computes an equilibrium point that balances the data fidelity against the fixed points of the denoiser. When the denoiser is locally homogeneous and has a symmetric Jacobian [29], the term corresponds to the gradient of the regularization function
(6) 
RED has been extensively applied in computational imaging, including in image restoration [28], phase retrieval [27], and tomographic imaging [1]. Later works have developed scalable variants of RED [26, 39] and further replace the AWGN denoising with a general artifactremoval operator [40].
2.3 Calibrating the Measurement Operator
The targetfree calibration of the measurement operator has been an emerging topic in recent years. For CT scanners with slightly misaligned projection angles, existing calibration methods include entropybased correction [41], frequencybased correction [42], and a technique that combined gradient descent and multirange testing [14]. A popular method to calibrate projection angles of CryoEM is called projectionmatching [43, 44], which relies on a prior estimate of the density map of the unknown single particle. It seeks the angle of every projection by pairing it with a clean template of the projection of the estimated density map from a known angle. Malhotraz et al. [16]
propose a threestep pipeline for CryoEM to recover the projection angles and the target density map in sequence: (a) denoise the projections by the Principal Component Analysis technique, (b) estimate the angles through iterative coordinate gradient descent, and (c) reconstruct the density map by
filtered backprojection (FBP). The key contribution of this paper is the investigation of the calibration problem in the context of PnP/RED algorithms, which has not been done in the prior work.3 Proposed Method
The proposed CalRED algorithm is summarized in Algorithm 1. It alternates between the updates of and . We initialize with their corresponding nominal values, , which are assumed to be known from the design of the imaging system. We additionally initialize the image with a direct inversion from measurements , which for CT reduces to the traditional FBP [9].
For the optimization, as shown in lines 57 in Algorithm 1, we minimize the objective function
(7) 
by performing the gradient descent
(8) 
In Equation 7, we consider the Tikhonov penalty as the prior for . The partial derivative is calculated by automatic differentiation [45].
We update by following the RED framework, as shown in lines 911 of Algorithm 1. For the denoising prior , we choose a 17layer DnCNN architecture in which all the convolution filters are of size and every feature map has 64 channels. Note that CalRED does not modify the update step of RED, which implies that the update step of CalRED could also be integrated within PnP or other variants of RED, such as BCRED [39].
CalRED relies on Nesterov acceleration [46] when computing both and updates by adding the acceleration sequence
(9) 
When for all , the algorithms reverts to the usual gradient method without acceleration. In Algorithm 1, the variables and store the intermediate values used for acceleration of and , respectively.
4 Experiments
In this section, we validate the performance of CalRED on simulated parallelbeam CT reconstruction, where the 2D target is rotated around an axis to acquire projections from different angles and form a sinogram. We assume that the CT machine is designed to project from nominal angles , an array of 90 projection angles that are evenlydistributed on a half circle. However, due to alignment errors, the CT machine actually projects from , another array of 90 angles that is assumed to be the result of corrupting
by an unknown AWGN vector with standard deviation (SD) of 1° and 5°, respectively. In our experiments, we synthesize the sinograms by using the groundtruth
and further corrupt these sinograms by AWGN corresponding to the input SNR of 40 dB and 30 dB, respectively.We quantify the image quality by using SNR (dB)
(10) 
and the angular error by RMSE
(11) 
All numerical results are averaged over six medical knee images discretized to the size of pixels. Note that the measurement operator in this case corresponds to the Radon transform.
We compare CalRED with several baseline methods, including RED, UNet, FISTA and the traditional leastsquare method (LSM). For both CalRED and RED, we train DnCNN denoisers for AWGN removal at three noise levels and pick the one that achieves the best performance for evaluation. We generate the training data for UNet by filteredbackprojecting the sinogram using the mismatched angles . We train an individual UNet for each combination of . In order to better demonstrate the benefit of using deep denoising prior, we run FISTA with the TV regularizer for comparison. We also incorporate the update step into both FISTA and LSM to construct the anglecalibrating FISTA (CalFISTA) and the anglecalibrating LSM (CalLSM), respectively.
Figure 1 illustrates the convergence of CalRED in terms of both image SNR and angular RMSE. In the cases where is severely mismatched, CalRED is able to reduce the angular RMSE by a factor of more than 7 — from 5°to around 0.65°; moreover, CalRED can achieve a good reconstruction comparable to that of RED using the true with respect to SNR. Table 1 and Table 2 summarize the numerical results under different combinations of input sinogram SNR and angular error. It is worth noting that CalRED and CalFISTA significantly outperform RED and FISTA, respectively, which highlights the effectiveness of the update step. Despite being designed for the angle calibration purpose, when CalRED is applied to a problem without calibration errors, it achieves an SNR only 0.03 dB less than that of the original RED. When the input SNR is 30 dB, the angles recovered by CalRED and CalFISTA are far more accurate than those recovered by CalLSM, which demonstrates how denoising priors can also contribute to measurement operator calibration.
Figure 2 compares the visual results of FBP, UNet, CalFISTA, CalRED, and RED. Both the reconstructions of FBP and uncalibrated RED suffer from obvious line artifacts, which demonstrates their vulnerability to a mismatched measurement operator. Although CalFISTA achieves an SNR close to that of CalRED, its reconstruction contains visible blocking artifacts. The UNet removes most of the artifacts that are contained in the FBP reconstruction, but it also has an undesirable effect of oversmoothing small features. While the traditional deep learning alone fails to balance the preservation of features and the removal of artifacts, deep denoising priors like DnCNN help CalRED to reconstruct highresolution images with minimal artifacts. For this particular image, when the projection angles are corrupted by AWGN of SD 5°, CalRED is able to achieve a remarkable SNR of 21.40 dB, while uncalibrated RED only achieves SNR of 10.01 dB.
Input SNR  30 dB  40 dB  
Angular Error  0°  1°  5°  0°  1°  5° 
FBP  14.17  12.69  9.09  19.42  15.68  10.14 
UNet  19.84  18.01  14.49  22.01  18.58  14.55 
FISTA  19.40  16.92  11.55  21.20  17.19  11.56 
RED  19.99  15.46  10.06  22.08  15.84  10.16 
CalLSM  7.80  5.07  4.18  16.73  14.88  13.34 
CalFISTA  19.35  19.14  18.69  21.17  20.84  20.29 
CalRED  19.96  19.74  19.52  22.07  21.78  21.13 
Input SNR  30 dB  40 dB  

Angular Error  1°  5°  1°  5° 
CalRED  0.073  0.650  0.024  0.651 
CalFISTA  0.078  0.649  0.029  0.648 
CalLSM  0.281  0.797  0.026  0.649 
5 Conclusion
In this letter, we propose CalRED, a method that takes advantage of advanced denoising priors to jointly recover the measurement operator parameters and the target image. We validate its robustness to severely mismatched measurement operator parameters and noisy measurements via simulated CT reconstruction. Future work includes extending the proposed calibration strategy to other applications and image reconstruction frameworks.
Acknowledgements
Research presented in this article was supported by the Laboratory Directed Research and Development program of Los Alamos National Laboratory under project number 20200061DR.
References
 [1] Z. Wu, Y. Sun, A. Matlock, J. Liu, L. Tian, and U. S. Kamilov, “SIMBA: Scalable inversion in optical tomography using deep denoising priors,” IEEE Journal of Selected Topics in Signal Process., vol. 14, no. 6, pp. 1163–1175, Mar. 2020.

[2]
S. Sreehari, S. V. Venkatakrishnan, B. Wohlberg, G. T. Buzzard, L. F. Drummy, J. P. Simmons, and C. A. Bouman, “Plugandplay priors for bright field electron tomography and sparse interpolation,”
IEEE Trans. Comput. Imaging, vol. 2, no. 4, pp. 408–423, Dec. 2016. 
[3]
Z. Wu, Y. Sun, J. Liu, and U. Kamilov, “Online regularization by denoising
with applications to phase retrieval,” in
IEEE Int. Conf. Computer Vision (ICCV) Workshops
, Oct. 2019.  [4] E. J. Candès, Y. C. Eldar, T. Strohmer, and V. Voroninski, “Phase retrieval via matrix completion,” SIAM J. Imaging Sci., vol. 6, no. 1, Feb. 2013.
 [5] H.Y. Liu, U. S. Kamilov, D. Liu, H. Mansour, and P. T. Boufounos, “Compressive imaging with iterative forward models,” in Proc. IEEE Int. Conf. Acoustics, Speech and Signal Process. (ICASSP), New Orleans, LA, USA, Mar. 2017.
 [6] C. A. Metzler, A. Maleki, and R. G. Baraniuk, “From denoising to compressed sensing,” IEEE Trans. Inf. Theory, vol. 62, no. 9, pp. 5117–5144, Sep. 2016.
 [7] A. Ribés and F. Schmitt, “Linear inverse problems in imaging,” IEEE Signal Process. Mag., vol. 25, no. 4, pp. 84–99, Jul. 2008.
 [8] L. I. Rudin, S. Osher, and E. Fatemi, “Nonlinear total variation based noise removal algorithms,” Physica D, vol. 60, no. 1–4, pp. 259–268, Nov. 1992.
 [9] A. C. Kak and M. Slaney, Principles of Computerized Tomographic Imaging. IEEE, 1988.
 [10] C. E. Cann, “Quantitative CT for determination of bone mineral density: a review.” Radiology, vol. 166, no. 2, pp. 509–522, 1988.
 [11] X. Wang, J. G. Mainprize, M. P. Kempston, G. E. Mawdsley, and M. J. Yaffe, “Digital breast tomosynthesis geometry calibration,” in Medical Imaging 2007: Physics of Medical Imaging, vol. 6510, 2007.
 [12] H. Miao, X. Wu, H. Zhao, and H. Liu, “A phantombased calibration method for digital Xray tomosynthesis,” Journal of Xray Science and Technology, vol. 20, pp. 17–29, Jan. 2012.
 [13] J. Zou, X. Hu, H. Lv, and X. Hu, “An investigation of calibration phantoms for CT scanners with tube voltage modulation,” Int. Journal of Biomedical Imaging, vol. 2013, Dec. 2013.
 [14] C. Cheng, Y. Ching, P. Ko, and Y. Hwu, “Correction of center of rotation and projection angle in synchrotron Xray computed tomography,” Scientific Reports, vol. 8, Dec. 2018.
 [15] D. Lee, P. Hoffmann, D. Kopperdhal, and T. Keaveny, “Phantomless calibration of CT scans for measurement of bmd and bone strengthinteroperator reanalysis precision,” Bone, vol. 103, Aug. 2017.

[16]
E. Malhotra and A. Rajwade, “Tomographic reconstruction from projections with unknown view angles exploiting momentbased relationships,” in
2016 IEEE Int. Conf. Image Process. (ICIP), 2016.  [17] S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg, “Plugandplay priors for model based reconstruction,” in Proc. IEEE Global Conf. Signal and Inf. Process. (GlobalSIP), 2013.
 [18] Y. Romano, M. Elad, and P. Milanfar, “The little engine that could: Regularization by denoising (RED),” SIAM J. Imaging Sci., vol. 10, no. 4, pp. 1804–1844, 2017.
 [19] S. H. Chan, X. Wang, and O. A. Elgendy, “Plugandplay ADMM for image restoration: Fixedpoint convergence and applications,” IEEE Trans. Comput. Imaging, vol. 3, no. 1, pp. 84–98, Mar. 2017.
 [20] A. M. Teodoro, J. M. BioucasDias, and M. Figueiredo, “A convergent image fusion algorithm using sceneadapted Gaussianmixturebased denoising,” IEEE Trans. Image Process., vol. 28, no. 1, pp. 451–463, Jan. 2019.
 [21] A. Brifman, Y. Romano, and M. Elad, “Turning a denoiser into a superresolver using plug and play priors,” in Proc. IEEE Int. Conf. Image Proc. (ICIP), Phoenix, AZ, USA, Sep. 2016.
 [22] A. M. Teodoro, J. M. BiocasDias, and M. A. T. Figueiredo, “Image restoration and reconstruction using variable splitting and classadapted image priors,” in Proc. IEEE Int. Conf. Image Proc. (ICIP), Phoenix, AZ, USA, Sep. 2016.

[23]
K. Zhang, W. Zuo, S. Gu, and L. Zhang, “Learning deep CNN denoiser prior for
image restoration,” in
Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)
, 2017.  [24] T. Meinhardt, M. Moeller, C. Hazirbas, and D. Cremers, “Learning proximal operators: Using denoising networks for regularizing inverse imaging problems,” in Proc. IEEE Int. Conf. Computer Vision (ICCV), Venice, Italy, Oct. 2017.
 [25] U. S. Kamilov, H. Mansour, and B. Wohlberg, “A plugandplay priors approach for solving nonlinear imaging inverse problems,” IEEE Signal Process. Lett., vol. 24, no. 12, pp. 1872–1876, Dec. 2017.
 [26] Y. Sun, B. Wohlberg, and U. S. Kamilov, “An online plugandplay algorithm for regularized image reconstruction,” IEEE Trans. Comput. Imaging, vol. 5, no. 3, pp. 395–408, 2019.

[27]
C. A. Metzler, P. Schniter, A. Veeraraghavan, and R. G. Baraniuk, “prDeep:
Robust phase retrieval with a flexible deep network,” in
Proc. 35th Int. Conf. Machine Learning (ICML)
, 2018.  [28] G. Mataev, M. Elad, and P. Milanfar, “DeepRED: Deep image prior powered by RED,” in Proc. IEEE Int. Conf. Computer Vision Workshops (ICCVW), Seoul, South Korea, Oct. 2019.
 [29] E. T. Reehorst and P. Schniter, “Regularization by denoising: Clarifications and new interpretations,” IEEE Trans. Comput. Imaging, vol. 5, no. 1, pp. 52–67, Mar. 2019.
 [30] K. Zhang, W. Zuo, Y. Chen, D. Meng, and L. Zhang, “Beyond a Gaussian denoiser: Residual learning of deep CNN for image denoising,” IEEE Trans. Image Process., vol. 26, no. 7, pp. 3142–3155, Feb. 2017.
 [31] M. V. Afonso, J. M.BioucasDias, and M. A. T. Figueiredo, “Fast image recovery using variable splitting and constrained optimization,” IEEE Trans. Image Process., vol. 19, no. 9, pp. 2345–2356, Sep. 2010.
 [32] A. Beck and M. Teboulle, “Fast gradientbased algorithm for constrained total variation image denoising and deblurring problems,” IEEE Trans. Image Process., vol. 18, no. 11, pp. 2419–2434, November 2009.

[33]
M. T. McCann, K. H. Jin, and M. Unser, “Convolutional neural networks for inverse problems in imaging: A review,”
IEEE Signal Process. Mag., vol. 34, no. 6, pp. 85–95, 2017. 
[34]
A. Lucas, M. Iliadis, R. Molina, and A. K. Katsaggelos, “Using deep neural networks for inverse problems in imaging: Beyond analytical methods,”
IEEE Signal Process. Mag., vol. 35, no. 1, pp. 20–36, Jan. 2018.  [35] F. Knoll, K. Hammernik, C. Zhang, S. Moeller, T. Pock, D. K. Sodickson, and M. Akcakaya, “Deeplearning methods for parallel magnetic resonance imaging reconstruction: A survey of the current approaches, trends, and issues,” IEEE Signal Process. Mag., vol. 37, no. 1, pp. 128–140, Jan. 2020.
 [36] O. Ronneberger, P. Fischer, and T. Brox, “UNet: Convolutional networks for biomedical image segmentation,” in Medical Image Computing and ComputerAssisted Intervention (MICCAI), 2015.
 [37] Y. Han, J. Kim, and J. C. Ye, “Differentiated backprojection domain deep learning for conebeam artifact removal,” IEEE Trans. Medical Imaging, 2020.
 [38] Y. Sun, Z. Xia, and U. S. Kamilov, “Efficient and accurate inversion of multiple scattering with deep learning,” Opt. Express, vol. 26, no. 11, pp. 14 678–14 688, May 2018.
 [39] Y. Sun, J. Liu, and U. S. Kamilov, “Block coordinate regularization by denoising,” in Advances in Neural Inf. Process. Systems 33, Vancouver, BC, Canada, Dec. 814, 2019.
 [40] J. Liu, Y. Sun, C. Eldeniz, W. Gan, H. An, and U. S. Kamilov, “RARE: Image reconstruction using deep priors learned without ground truth,” IEEE J. Select. Topics Signal Process., 2020.
 [41] T. Donath, F. Beckmann, and A. Schreyer, “Automated determination of the center of rotation in tomography data,” Journal of the Optical Society of America. A, Optics, Image science, and Vision, vol. 23, pp. 1048–57, Jun. 2006.
 [42] N. T. Vo, M. Drakopoulos, R. Atwood, and C. Reinhard, “Reliable method for calculating the center of rotation in parallelbeam tomography,” Optics Express, vol. 22, pp. 19 078–19 086, Jul. 2014.
 [43] P. A. Penczek, R. A. Grassucci, and J. Frank, “The ribosome at improved resolution: New techniques for merging and orientation refinement in 3D cryoelectron microscopy of biological particles,” Ultramicroscopy, vol. 53, no. 3, pp. 251 – 270, 1994.
 [44] T. S. Baker and R. H. Cheng, “A modelbased approach for determining orientations of biological macromolecules imaged by cryoelectron microscopy,” Journal of Structural Biology, vol. 116, no. 1, pp. 120 – 130, 1996.
 [45] A. Baydin, B. Pearlmutter, A. Radul, and J. Siskind, “Automatic differentiation in machine learning: A survey,” Journal of Machine Learning Research, vol. 18, pp. 1–43, Apr. 2018.
 [46] Y. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course. Kluwer Academic Publishers, 2004.
Comments
There are no comments yet.