1 Introduction
As a classical and fundamental problem in low level vision, image denoising has been extensively explored and it is still under investigation as well in methodological aspects. In general, image denoising aims to recover the latent noisefree image from its noisecorrupted version. During the past decades, image denoising has got great development in a variety of models including spatial filtering [31], total variation [29], wavelet/curvelet transform [14], noise detector [3, 6, 13], sparse representation [4, 12], nonlocal means and its varieties [1, 8, 10, 19, 39], deep learning[9, 20, 30, 36, 37], etc.
The landmark work of image denoising is nonlocal means (NLM) [1] which brings a new era by finding nonlocal similar patches within the image. Quite a few patchbased denoising methods [8, 10, 12, 19, 32, 37] achieve great success by exploiting the internal selfsimilarity prior. A quintessential example should be cited that BM3D [10] uses 3D filtering in transformed domain by patch matching, which is quite effective and has been a benchmark in image denoising. Here, patch matching is performed within a relatively large local window instead of globally, and consequently the number of patches may not be sufficient that leads to ringing artifacts. In order to improve the accuracy of patch matching, external natural patch priors are used to regularize the denoising process. Method [7] learns the covariance matrix and the measure of redundancy for each patch based on clustered natural noisefree patch database. Zoran and Weiss [40]
gave high likelihood values for patches sampled from clean images using Gaussian mixture models (GMM), and reconstructed the latent image by maximizing the expected patch log likelihood. Though external patch priors based approaches have achieved remarkable success in image restoration, they are still local and do not make full use of image internal selfsimilarity. Some methods
[12, 19] globally search for the most similar patches across the whole image and have demonstrated competitive results. Though global patch matching is to determine enough similar patches, the error rate of patch matching may be higher, which results in destroying image details unsatisfactorily. Methods PCLR [8] and PGPD [35] integrate external patch priors and internal selfsimilarity into one framework. They learn GMM prior information from noisefree image patches to guide the noisy patches classification and the subsequent low rank subspace learning, which makes the latent patch reconstruction more robust to noise. However, there is no obvious improvement in denoising performance, compared with local patch matching methods. We argue that such utilization of GMM prior information is not effective enough because it neglects the noise of degraded image, which interferes with the selection of similar patches. How to precisely select similar patches is still an open problem.Besides, low rank matrix approximation (LRMA), which aims to recover the underlying low rank matrix from its degraded observation, appears in a wide range of applications in various fields including computer vision, machine learning, signal processing and bioinformatics. For instance, use of low rank approximation can be found in applications such as face recognition
[17, 11], background modeling and foreground object detection [19, 21, 15], image alignment [27, 38], video denoising [22, 28] image restoration [8, 33, 12], among others. Since direct rank minimization is difficult to solve, the problem is generally relaxed by substitutively minimizing the nuclear norm of the estimated matrix
[16]. The nuclear norm of a matrix , denoted by, is defined as the sum of its singular values, i.e.
, where denotes the th singular value of . The nuclear norm minimization model has been attracting significant attention due to its rapid development in both theory and implementation. Cai et al. [2] proved that the model can be easily solved in a closed form by imposing a soft thresholding operation on the singular values of the observation matrix. Candès et al. [5]proved that from the noisy input, its intrinsic lowrank reconstruction can be exactly achieved with a high probability by the model. Many stateoftheart denoising methods
[18, 34, 25, 12] have been proposed in recent years by utilizing nuclear norm proximal as the key proximal technique. Albeit its success as mentioned before, nuclear norm minimization model still has certain limitations. Gu et al. [19] proposed weighted nuclear norm minimization model trying to achieve a better estimation of the latent data from the corrupted input. The weighted nuclear norm of a matrix is given by , where and is a nonnegative weight assigned to . The nuclear norm minimization model provides superior performance and efficiency for matrix completion or impulse noise. However, they ignore the difference between Gaussian noise and impulse noise, then building a low rank model for Gaussian noise is still a worthwhile work.This article will address two issues: (1) how to improve similar patches matching accuracy; (2) build an appropriate low rank matrix approximation model for Gaussian noise.
Firstly, unlike PCLR [8] and PGPD [35] clustering the noisy patches directly, we first carry out a preliminary denoising of the image by a local patch matching based denoising algorithm and get a preprocessed image, then cluster all preprocessed patches into
classes. If a class contains too many similar patches, we use kmeans to divide the big class into some smaller ones. We next record all patches location information. Consequently, in order to classify noisy patches, we replace all preprocessed patches by the corresponding noisy ones in each class according to the location information. Furthermore, the preprocessed image is updated in each iteration in order to improve patch matching accuracy. This idea significantly increases the accuracy of similar patch clustering. Here, we use a local patch matching based denoising method because it matches similar patches with higher precision. Since BM3D
[10] is the stateoftheart local patch matching based method and a realtime computing algorithm, we use it as a preprocessing algorithm in this paper. Though some global patch matching based methods outperform BM3D, they introduce some errors for patch matching and the errors can’t be corrected in the future denoising steps. In addition, the methods PCLR [8] and PGPD [35] are iterative algorithms. This means that the methods PCLR [8] and PGPD [35] have already used a global patch matching based algorithm (the th step denoising) as a preprocessing algorithm for the th step denoising.Secondly, according to the characteristics of Gaussian noise, the minimization of i.e. the square of the Frobenius norm of adapts Gaussian noise model. Inspired by [39] and the equation , similar to nuclear norm minimization model [2], we will establish Gaussian nuclear norm minimization model by minimizing and then efficiently solve the problem.
As noted above, the proposed method uses GMM to achieve both local and global patch matching, and the mixture of patches can be further regularized via kmeans clustering procedure and low rank minimization. This process may seem complicated, but it is very effective. In the low rank denoising part, we use the soft threshold given by [19] which is good to smooth the image. The proposed model is solved by an alternating minimization procedure and it converges rapidly.
The rest of the paper is organized as follows. In Section 2, we propose our algorithm by using GMM so that both local and global patch matching can be achieved. The complexity and convergence of the algorithm are also been discussed. Section 3 shows experiment results and discussion. Conclusions and future works are given in Section 4.
2 The Proposed Algorithm
Throughout this paper, we denote scalars, vectors and matrices by non boldfaced, boldfaced lowercase, and boldfaced uppercase letters, respectively.
2.1 The low rank regularization model based on GMM prior
The noisecorrupted images are often represented by the following simplified model:
(1) 
where is noisy image, is latent original image and is additive white Gaussian (AWG) noise matrix of mean
. Let and be the vectorized image patches which are extracted from the noisy image and latent original image at the th pixel, respectively. All noisy patches are divided into similar patch groups. Consequently, these similar patches in th group are collected to form a reshaped matrix , where may be different for each . The noise model (1) can be rewritten in the patchbased representation as follow:(2) 
where denote the patch reshaped matrices from the noisy image , original image and noise matrix , respectively.
In this paper, we will introduce preprocessed image based GMM prior low rank regularization model (PGLR):
(3) 
where is the Frobenius norm, denotes nuclear norm. is preprocessed image based patch cluster log likelihood term, is the th patch extracted from preprocessed image , and denotes which class is from. Our model effectively incorporates image internal selfsimilarity and external patch priors into the low rank denoising model. There are two advantages of such clustering based low rank regularization.
First, the Gaussian patch mixture model based patch clustering can take the advantages of local and global patch matching. Local patch matching is to find similar patches in a large neighborhood which can alleviate noise effect, while the number of patches may not be sufficient. Global patch matching aims to determine enough similar patches but the error rate of patch matching may be higher. The model first use local patch matching method to reduce noise and then use Gaussian patch mixture model to achieve global patch matching. The detail will be discussed in Subsection 2.2.
Second, the nuclear norm minimization model [2] is an outstanding model for removing impulse noise and inpainting, but not adapt to Gaussian noise. Here according to the statistic properties of Gaussian noise and the nuclear norm minimization model [2], we build a new model for Gaussian noise: , where is the number of rows in the matrix . We prove that there is a global optimal solution of the model and the simulation experiment shows that our model is much better than the nuclear norm minimization model [2] for Gaussian noise (see the supplementary material). Subsection 2.3 will give details.
2.2 Improved Gaussian Patch Mixture Model
In this subsection, we first recall GMM model [40]. All noisefree image patches , () are divided into groups with a parameter , , which describes a Gaussian density function parameterized by the mean and covariance matrix . Thus, the probability of each patch could be defined as a weighted sum of Gaussians:
(4) 
where
(5) 
and is the transpose of , and is the set of parameters with and , .
We investigate the model (4) and discover that is unknown latent image patch and consequently it’s impossible to calculate . In the model (4), it normally takes noisy patch instead of noisefree patch . As a result, the error rate of patch clustering increases significantly due to noise. In this paper, we will solve the problem.
The latent image patch is unknown and the noisy image patch is unfavorable, then we should find another solution. There are quite a few excellent denoising algorithms [1, 8, 10, 19], and a distinguished one is BM3D [10]. BM3D is a realtime computing algorithm and gets favorable restoration results. Furthermore, it matches similar patches in a large neighborhood, which alleviates noise effect for selection of similar patches. We take BM3D as a image preprocessing algorithm and get a preprocessed image that is a rosy estimate of the noisefree one. For this reason, the preprocessed image based patch cluster log likelihood (PCLL) term is given as follow:
(6) 
Obviously, our model is feasible and efficient as we know is the pleasurable estimate of . Consequently, we cluster all preprocessed patches into classes by maximizing the PCLL term. However, the number of image patches varies greatly from class to class. In order to equalize the number of similar patches in each class, we further adjust the classification. If a class doesn’t contain enough similar image patches, it will be merged with the nearest one. If there are excessive patches in a class, kmeans is used to divide the large class into some finer ones. Instead of constructing low rank matrices, these preprocessed image patches in each class are used to record patches location information. Finally, all the noisy image patches are clustered by the guide of the location information of the corresponding preprocessed ones. This clustering process is somewhat complex, but the accuracy is undoubtedly higher. As a result, the rank of the created matrix by stacking patches from each class is lower.
Note: why not we use a denoising algorithm based on global patch matching? Jin et al. [23, 24] have proved that nonlocal means is a local method with a large neighborhood and there is an optimal size of search window. Though global patch matching aims to acquire enough similar patches, the error rate of patch matching is intensely higher because of noise interference. In the image preprocessing, if an image patch is classified into a wrong class, the error will not be corrected in later process. For BM3D [10], the number of patches may be insufficient and the result may not as good as global patch matching based methods (such as WNNM [19], PCLR [8] and PGPD [35]), but the error rate of patch matching is lower. The restoration result will be improved in the future denoising steps but the error will exist forever. Another reason for not using global patch matching based methods is that they run slowly.
2.3 Gaussian nuclear norm minimization for low rank modeling
Consider the singular value decomposition (SVD) of a matrix
of rank :where and are respectively and matrices with orthonormal columns and the singular values are positive and satisfy that .
Nuclear norm minimization (NNM) [16] is an effective rank minimization. The nuclear norm of a matrix , denoted by , is defined as the sum of its singular values, i.e.
(7) 
The NNM approach has been attracting significant attention due to its rapid development in both theory and implementation. The nuclear norm proximal (NNP) problem [2] is
(8) 
where is a positive constant.
Theorem A (Cai et al. [2]) For each , the global optimum of model (8) is
(9) 
where denotes the th singular value of and .
Albeit its success as aforementioned, NNM ignores the prior knowledge we often have about singular values of a practical data matrix. More specifically, larger singular values of an input data matrix quantify the information of its underlying principal directions. In order to improve the flexibility of NNM, Gu et al. [19] proposed the weighted nuclear norm. The weighted nuclear norm of a matrix is defined as
(10) 
where is a nonnegative weight assigned to . The weight vector will enhance the representation capability of the original nuclear norm. The weighted nuclear norm proximal (WNNP) problem is given by
(11) 
Theorem B (Gu et al. [19]) If the weight satisfies , the global optimum of model (11) is
(12) 
The WNNM model is more difficult to optimize than conventional NNM ones due to the nonconvexity of the involved weighted nuclear norm.
In this paper, we consider the proprieties of Gaussian noise. By tradition, the LRMA for Gaussian noise is to minimize . Note that , we adapt NNM model (8) to Gaussian noise by utilizing to take place of . Then we have Gaussian nuclear norm minimization (GNNM):
(13) 
We prove that the Gaussian nuclear norm minimization problem can be equivalently transformed to a quadratic programming (QP) problem with linear constraints. This allows us to easily reach the global optimum of the original problem by using offtheshelf convex optimization solvers. In order to solve the problem (13), we introduce the following theorem.
Theorem 1 For each , the global optimum of model (13) is
(14) 
Proof Considering as a new matrix and using Theorem A, we have that the global optimum of model (13) is , where denotes the th singular value of . Let and , it is easy to get . Then we have
Therefore, is also the global optimum of model (13).
A image contaminated by AGW noise i.e. model (1) can be recovered by the Theorem 1. We next discuss the parameter . If the standard deviation of AWG noise is , we have the shrinkage amount . According to the AGW noise model (1
), we deduce the relationship between the eigenvalues of the covariance matrix for the noisy observation
and those for its latent clean image . Sinceis assumed to follow a Gaussian distribution with zero mean and variance
, the eigenvalue decomposition of the covariance matrix can be expressed as2.4 Optimization and algorithm
We first divide all noisefree image patches into classes and learn the parameter by GMM. Then there are only three unknowns in the proposed model (2.1): the latent image , class label and reshaped low rank matrices , . An alternating minimization procedure is used to solve our model. We start with the preprocessed image . In the th iteration, the solutions to the alternating minimization scheme are detailed as follows.
(1) Fix , solve for the class label .
For each patch
, we use the probability density function of the learned GMM to calculate the likelihood within each class,
(17) 
Then the with the maximum probability is assigned to .
(2) Fix , we calculate low rank matrices , , according to
(18) 
Like Gu et al. [19], we solve the low rank denoising model (2.1) by the formulas as follows:
(19)  
here is the singular value matrix of , denotes the singular value decomposition (SVD) of , is the number of similar patches, and is the variance of image in the th iteration.
(3) Fix , , a weighted averaging method is used to reconstruct the estimated image by aggregating all denoised patches. The value of the th pixel in the reference image is calculated as a weighted average as follows,
(20) 
where refers to the denoised image intensity of the th pixel in the th patch of th patch group , and the summation is carried out over all patches that overlap with the th pixel. The empirical weight is given as
(21) 
is the rank of matrix , is the number of similar patches in the th class.
In addition, at the beginning of each iteration , the desired denoised image, preprocessed image and standard deviation are updated by
(22) 
(23) 
(24) 
where means the th iteration, is the iteration regularization parameter, is the noisy image, is the denoised image in the th iteration, is the noise variance of the input noisy image, and is the scale factor of controlling the reestimation noise standard deviation . The complete optimization process is shown in Algorithm 1.
2.5 Complexity and convergence
Suppose that there are patches extracted from the whole image and pixels in each patch. The main computational cost in each iteration is mainly composed of three parts. Firstly, classifying patches into classes costs , where the covariance matrix determinant needs calculations. Next, the computation of the low rank matrix reconstruction by SVD for clusters takes . Finally, averaging all the grouping similar patches needs computations. Thus the computation complexity is and the classification step takes most computations of the algorithm.
(a)  (b) 
Experience shows our method has a rapid convergence and reaches the optimal solution at the th or th iteration although it is nonconvex. Figure 1 compares the convergence speed between our algorithm and OLRA [39]. In the comparative experiment, the Cameraman image with intricate details was selected. Figure 1 (a) and (b) describe the variation curve of PSNR and SSIM values, respectively. It’s obvious that our method is always superior than OLRA [39] after the th iteration. As shown in Table 1, our algorithm is almost the fastest among all low rank based methods.
Image size  

BM3D[10]  1.26  3.22 
WNNM [19]  117.97  677.67 
PCLR [8]  196.11  1090.40 
OLRA [39]  202.20  1014.63 
DnCNN [36]  18.83  15.93 
FFDNet [37]  16.49  17.23 
Ours  129.45  614.10 
Cameraman  Monarch  House  Peppers  Montage  Lena  Barbara  Boat  Couple  Hill 
Traditional algorithm  Deep Learning  
PSNR/SSIM  BM3D [10]  WNNM [19]  PCLR [8]  OLRA [39]  Ours  FFDNet [37]  DnCNN [36]  DnCNN*[36]  
25  Cameraman  29.45/0.854  29.63/0.860  29.66/0.863  29.58/0.863  29.90/0.870  30.05/0.877  30.18/0.876  28.87/0.823 
Monarch  29.25/0.890  29.85/0.909  29.75/0.908  29.92/0.912  30.26/0.918  30.14/0.917  30.23/0.917  29.26/0.879  
House  32.86/0.859  33.23/0.861  32.99/0.859  33.25/0.863  33.50/0.864  33.27/0.862  33.08/0.860  31.65/0.825  
Peppers  30.16/0.868  30.40/0.873  30.46/0.875  30.35/0.874  30.76/0.881  30.79/0.884  30.84/0.881  29.49/0.840  
Montage  32.37/0.926  32.74/0.933  32.36/0.930  32.26/0.929  32.74/0.934  32.84/0.941  32.95/0.937  30.68/0.879  
Lena  32.08/0.861  32.25/0.866  32.17/0.863  32.30/0.867  32.55/0.868  32.59/0.874  32.44/0.869  31.12/0.824  
Barbara  30.72/0.874  31.24/0.898  30.65/0.887  31.20/0.897  31.07/0.894  29.98/0.879  30.00/0.878  26.78/0.785  
Boat  29.91/0.801  29.83/0.795  30.00/0.800  29.95/0.803  30.34/0.811  30.23/0.812  30.21/0.809  29.45/0.780  
Couple  29.72/0.820  29.82/0.820  29.73/0.814  29.77/0.819  30.11/0.826  30.18/0.835  30.12/0.830  29.08/0.788  
Hill  29.85/0.775  29.95/0.776  29.84/0.771  29.94/0.778  30.24/0.785  30.04/0.782  30.03/0.780  29.43/0.758  
Average  30.64/0.855  30.91/0.860  30.76/0.857  30.85/0.861  31.15/0.865  31.01/0.866  31.01/0.864  29.58/0.818  
50  Cameraman  26.12/0.782  26.41/0.785  26.55/0.795  26.35/0.780  26.83/0.794  27.02/0.811  27.00/0.805  25.75/0.704 
Monarch  25.82/0.820  26.32/0.835  26.25/0.837  26.21/0.835  26.67/0.849  26.88/0.854  26.76/0.846  25.46/0.767  
House  29.69/0.812  30.33/0.823  29.77/0.819  30.19/0.816  30.40/0.818  30.43/0.829  30.01/0.820  27.98/0.711  
Peppers  26.68/0.794  26.91/0.801  27.03/0.808  27.06/0.800  27.69/0.820  27.43/0.820  27.29/0.810  25.93/0.724  
Montage  27.90/0.861  28.27/0.875  28.20/0.877  28.34/0.872  28.95/0.883  29.18/0.898  28.95/0.882  26.85/0.772  
Lena  29.05/0.799  29.25/0.806  29.12/0.801  29.01/0.798  29.68/0.811  29.68/0.823  29.36/0.811  27.78/0.705  
Barbara  27.23/0.795  27.79/0.820  27.11/0.798  27.65/0.811  27.90/0.821  26.48/0.779  26.23/0.770  24.08/0.642  
Boat  26.78/0.705  26.97/0.708  26.99/0.708  26.84/0.701  27.47/0.721  27.32/0.726  27.19/0.718  26.30/0.660  
Couple  26.46/0.707  26.64/0.714  26.55/0.706  26.65/0.714  27.16/0.727  27.07/0.737  26.89/0.724  25.83/0.651  
Hill  27.19/0.675  27.33/0.676  27.23/0.672  27.24/0.671  27.85/0.690  27.55/0.688  27.44/0.683  26.55/0.631  
Average  27.29/0.775  27.62/0.784  27.48/0.782  27.55/0.780  28.06/0.793  27.90/0.796  27.71/0.787  26.25/0.697  
75  Cameraman  24.33/0.734  24.55/0.735  24.74/0.747  24.65/0.741  25.16/0.753  25.29/0.770  25.09/0.759  22.48/0.466 
Monarch  23.91/0.756  24.31/0.775  24.28/0.781  24.10/0.776  24.71/0.796  24.99/0.801  24.70/0.783  22.34/0.588  
House  27.51/0.765  28.24/0.789  27.57/0.785  28.38/0.795  28.78/0.803  28.43/0.796  27.83/0.776  23.82/0.466  
Peppers  24.73/0.737  24.92/0.742  25.06/0.762  24.95/0.748  25.67/0.768  25.39/0.769  25.15/0.752  22.83/0.526  
Montage  25.52/0.800  25.72/0.821  25.70/0.827  25.56/0.831  26.57/0.851  26.84/0.857  26.30/0.829  23.36/0.516  
Lena  27.26/0.752  27.54/0.766  27.41/0.766  25.96/0.766  28.16/0.776  27.57/0.767  27.97/0.786  24.03/0.471  
Barbara  25.12/0.711  25.81/0.749  25.07/0.722  25.68/0.740  26.03/0.753  24.24/0.685  23.62/0.654  21.95/0.462  
Boat  25.12/0.641  25.30/0.647  25.29/0.649  25.00/0.634  25.78/0.658  25.64/0.666  25.47/0.655  23.01/0.461  
Couple  24.70/0.626  24.86/0.637  24.80/0.629  24.71/0.627  25.43/0.651  25.29/0.662  24.99/0.640  22.87/0.466  
Hill  25.68/0.612  25.88/0.619  25.82/0.616  25.59/0.604  26.36/0.625  26.15/0.633  25.95/0.621  23.59/0.454  
Average  25.39/0.713  25.71/0.728  25.57/0.728  25.46/0.726  26.27/0.743  26.02/0.742  25.67/0.724  23.03/0.487  
100  Cameraman  23.07/0.692  23.36/0.697  23.48/0.715  23.32/0.700  23.96/0.721  23.96/0.740  –  – 
Monarch  22.52/0.702  22.95/0.726  22.93/0.737  22.86/0.727  23.62/0.758  23.61/0.759  –  –  
House  25.87/0.720  26.66/0.754  25.97/0.750  26.66/0.755  27.22/0.770  26.91/0.768  –  –  
Peppers  23.39/0.688  23.45/0.698  23.70/0.725  23.66/0.711  24.45/0.743  23.96/0.733  –  –  
Montage  23.89/0.747  24.16/0.777  24.23/0.794  23.86/0.783  25.07/0.818  25.12/0.824  –  –  
Lena  25.95/0.709  26.21/0.726  26.16/0.736  26.96/0.725  27.02/0.741  26.77/0.756  –  –  
Barbara  23.62/0.643  24.37/0.686  23.65/0.660  24.10/0.672  24.63/0.691  22.82/0.614  –  –  
Boat  23.97/0.594  24.11/0.598  24.16/0.607  23.83/0.594  24.77/0.618  24.53/0.624  –  –  
Couple  23.51/0.567  23.56/0.570  23.65/0.575  23.55/0.567  24.37/0.597  24.05/0.600  –  –  
Hill  24.58/0.565  24.76/0.572  24.83/0.576  24.53/0.562  25.45/0.587  25.15/0.591  –  –  
Average  24.04/0.663  24.36/0.680  24.28/0.688  24.23/0.680  25.06/0.704  24.69/0.701  –  –  
(a) Original  (b) Noisy image  (c) BM3D[10] 27.51/0.765  (d) WNNM[19] 28.24/0.789  (e) PCLR[8] 27.57/0.785 
(f) OLRA[39] 28.38/0.795  (g) FFDNet[37] 28.43/0.796  (h) DnCNN[36] 27.83/0.776  (i) DnCNN*[36] 23.82/0.466  (j) Ours 28.78/0.803 
(a) Original  (b) Niosy image  (c) BM3D[10] 23.07/0.692  (d) WNNM[19] 23.36/0.697 
(e) PCLR [8] 23.48/0.715  (f) OLRA[39] 23.32/0.700  (g) FFDNet[37] 23.96/0.740  (h) Ours 23.96/0.721 
3 Experimental results and discussion
From the BSD dataset [26], 200 training images are uniformly sampled to image patches, where GMM with 250 mixing components and parameter are learned. The related initialization parameters are set to , , and the number of iterations .
The test images are selected from the BSD dataset [26] (see Figure 2). There are five images and five images. Our algorithm PGLR is compared with several stateoftheart algorithms, including BM3D [10], WNNM [19], PCLR [8], OLRA [39], DnCNN [36] and FFDNet [37]. All source codes are got from the authors.
Quantitatively, we evaluate the quality of image restoration by the peak signal to noise ratio (PSNR) and structural similarity index (SSIM). The SSIM is defined by Wang et al.
[33] and PSNR is given bywhere
is the image size, is the original image and is the restored image.
We record the values of PSNR / SSIM of restored image at various Gaussian noise level in Table 2. The table confirms that our method achieves the best results in almost all the cases in both PSNR and SSIM values among the traditional denoising algorithms and is also a little better than some deep learning based methods. Furthermore, our algorithm wins BM3D [10] more than in average when .
For visual image quality, Figures 3 and 4 show that the proposed method yields the best image visual quality in terms of removing noise, preserving edges and maintaining image details. Taking Figure 3 for example, the part zoomed in shows that our proposed method obtains pleasurable result at window frame while BM3D blurs the window. The low rank based algorithms (WNNM [19], PCLR [8] and OLRA [39]) introduce unpleasant artifacts, and the deep learning based methods cut the structure at the lower left corner of the window.
When the noise level is extremely high, the structural characteristics of the image are severely damaged and the restoration task becomes much arduous. In Figure 4, the cameraman’s ear is apparently restored almost completely by our algorithm, while other methods introduce serious artifacts in the hair and destroy the ear. FFDNet [37] sharpens the edges but removes some structures partly such as the details of the camera. Though our approach blurs the edges a little bit, it preserves most of the structures. In summary, our algorithm remains more detail information such as structures and edges than other advanced methods.
4 Conclusion
In this paper, we propose a novel method by combining Gaussian patch mixture model and low rank matrix restoration. We conduct extensive experiments at different Gaussian noise levels, showing that the proposed approach is robust and outperforms the stateoftheart in image denoising. The denoising capability of our proposed method largely benefits from the accuracy of similar patches selection, thereby it makes up for the lack of ability for low rank approximation and maximum expectation algorithms to maintain the details and edges. Future works include: (1) Global patch matching greatly increases the complexity by spreading over all image patches to find their similar ones. How to reduce the computational complexity and ensure great denoising performance is worth discussing. (2) The parameters in the simulation experiment are selected according to experience. The slight changes of a parameter will also make difference. Therefore, how to set an adaptive parameter function is also a point worth studying. (3) In the process of experiments, we find that when performing the classification operation, some classes have only a few similar patches while some others get so many ones. It is rational to set appropriate classes to balance the number of similar patches in each class, thereby enhancing the robustness of the algorithm. (4) The future work also includes extensions to other types of noise such as Poisson noise, mixed noise and unknown type of noise.
Acknowledgments
Jin has been supported by the National Natural Science Foundation of China (Grant No. 12061052), Natural Science Fund of Inner Mongolia Autonomous Region (Grant No. 2020MS01002), China Scholarship Council for a one year visiting at Ecole normale supérieure ParisSaclay (No. 201806810001).
References

[1]
Antoni Buades, Bartomeu Coll, and JM Morel.
A nonlocal algorithm for image denoising.
In
IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
, volume 2, pages 60–65, 2005.  [2] JianFeng Cai, Emmanuel J Candès, and Zuowei Shen. A singular value thresholding algorithm for matrix completion. SIAM Journal on Optimization, 20(4):1956–1982, 2010.
 [3] JianFeng Cai, Raymond H Chan, and Carmine Di Fiore. Minimization of a detailpreserving regularization functional for impulse noise removal. Journal of Mathematical Imaging and Vision, 29(1):79–91, 2007.
 [4] JianFeng Cai, Hui Ji, Zuowei Shen, and GuiBo Ye. Datadriven tight frame construction and image denoising. Applied and Computational Harmonic Analysis, 37(1):89–105, 2014.

[5]
Emmanuel J Candès, Xiaodong Li, Yi Ma, and John Wright.
Robust principal component analysis?
Journal of the ACM (JACM), 58(3):1–37, 2011.  [6] Raymond H Chan, Yiqiu Dong, and Michael Hintermuller. An efficient twophase tv method for restoring blurred images with impulse noise. IEEE Transactions on Image Processing, 19(7):1731–1739, 2010.
 [7] Priyam Chatterjee and Peyman Milanfar. Learning denoising bounds for noisy images. In IEEE International Conference on Image Processing (ICIP), pages 1157–1160, 2010.
 [8] Fei Chen, Lei Zhang, and Huimin Yu. External patch prior guided internal clustering for image denoising. In IEEE International Conference on Computer Vision (ICCV), pages 603–611, 2015.

[9]
Jingwen Chen, Jiawei Chen, Hongyang Chao, and Yang Ming.
Image blind denoising with generative adversarial network based noise modeling.
In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018.  [10] Kostadin Dabov, Alessandro Foi, Vladimir Katkovnik, and Karen Egiazarian. Image denoising by sparse 3d transformdomain collaborative filtering. IEEE Transactions on Image Processing, 16(8):2080–2095, 2007.
 [11] Jiayu Dong, Huicheng Zheng, and Lina Lian. Lowrank laplacianuniform mixed model for robust face recognition. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 11889–11898, 2019.
 [12] Weisheng Dong, Guangming Shi, and Xin Li. Nonlocal image restoration with bilateral variance estimation: a lowrank approach. IEEE Transactions on Image Processing, 22(2):700–711, 2012.
 [13] Yiqiu Dong, Raymond H Chan, and Shufang Xu. A detection statistic for randomvalued impulse noise. IEEE Transactions on Image Processing, 16(4):1112–1120, 2007.
 [14] David L Donoho. Denoising by softthresholding. IEEE Transactions on Information Theory, 41(3):613–627, 1995.
 [15] N Benjamin Erichson, Steven L Brunton, and J Nathan Kutz. Compressed dynamic mode decomposition for background modeling. Journal of RealTime Image Processing, 16(5):1479–1492, 2019.
 [16] Sarjoui M Fazel. Matrix rank minimization with applications. 2003.
 [17] Guangwei Gao, Jian Yang, XiaoYuan Jing, Fumin Shen, Wankou Yang, and Dong Yue. Learning robust and discriminative lowrank representations for face recognition with occlusion. Pattern Recognition, 66:129–143, 2017.
 [18] Qi Ge, XiaoYuan Jing, Fei Wu, ZhiHui Wei, Liang Xiao, WenZe Shao, Dong Yue, and HaiBo Li. Structurebased lowrank model with graph nuclear norm regularization for noise removal. IEEE Transactions on Image Processing, 26(7):3098–3112, 2016.
 [19] Shuhang Gu, Lei Zhang, Wangmeng Zuo, and Xiangchu Feng. Weighted nuclear norm minimization with application to image denoising. In IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pages 2862–2869, 2014.
 [20] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Sun Jian. Deep residual learning for image recognition. In IEEE Conference on Computer Vision and Pattern Recognition(CVPR), pages 770–778, 2016.
 [21] Sajid Javed, Arif Mahmood, Thierry Bouwmans, and Soon Ki Jung. Spatiotemporal lowrank modeling for complex scene background initialization. IEEE Transactions on Circuits and Systems for Video Technology, 28(6):1315–1329, 2016.
 [22] Hui Ji, Chaoqiang Liu, Zuowei Shen, and Yuhong Xu. Robust video denoising using low rank matrix completion. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1791–1798, 2010.
 [23] Qiyu Jin, Ion Grama, Charles Kervrann, and Quansheng Liu. Nonlocal means and optimal weights for noise removal. SIAM Journal on Imaging Sciences, 10(4):1878–1920, 2017.
 [24] Qiyu Jin, Ion Grama, and Quansheng Liu. Convergence theorems for the nonlocal means filter. Inverse Problems & Imaging, 12(4):853–881, 2018.
 [25] Zhouchen Lin, Risheng Liu, and Zhixun Su. Linearized alternating direction method with adaptive penalty for lowrank representation. In Advances In Neural Information Processing Systems (NIPS), pages 612–620, 2011.
 [26] David Martin, Charless Fowlkes, Doron Tal, and Jitendra Malik. A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In IEEE International Conference on Computer Vision (ICCV), volume 2, pages 416–423. IEEE, 2001.
 [27] Yigang Peng, Arvind Ganesh, John Wright, Wenli Xu, and Yi Ma. Rasl: Robust alignment by sparse and lowrank decomposition for linearly correlated images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 34(11):2233–2246, 2012.
 [28] Xutong Ren, Wenhan Yang, WenHuang Cheng, and Jiaying Liu. Lr3m: Robust lowlight enhancement via lowrank regularized retinex model. IEEE Transactions on Image Processing, 29:5862–5876, 2020.
 [29] Leonid I. Rudin, Stanley Osher, and Emad Fatemi. Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenomena, 60(14):259–268, 1992.
 [30] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for largescale image recognition. arXiv preprint arXiv:1409.1556, 2014.
 [31] C. Tomasi and R. Manduchi. Bilateral filtering for gray and color images. In IEEE International Conference on Computer Vision (ICCV), pages 839–846, 1998.
 [32] Hang Wang, Siyuan Cao, Kangkang Jiang, Hao Wang, and Qingchen Zhang. Seismic data denoising for complex structure using bm3d and local similarity. Journal of Applied Geophysics, 170, 2019.
 [33] Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity. IEEE Transactions on Image Processing, 13(4):600–612, 2004.

[34]
Zhaojun Wu, Qiang Wang, Jing Jin, and Yi Shen.
Structure tensor total variationregularized weighted nuclear norm minimization for hyperspectral image mixed denoising.
Signal Processing, 131:202–219, 2017.  [35] Jun Xu, Lei Zhang, Wangmeng Zuo, David Zhang, and Feng Xiangchu. Patch group based nonlocal selfsimilarity prior learning for image denoising. In IEEE International Conference on Computer Vision (ICCV), pages 244–252, 2015.
 [36] Kai Zhang, Wangmeng Zuo, Yunjin Chen, Deyu Meng, and Lei Zhang. Beyond a gaussian denoiser: Residual learning of deep cnn for image denoising. IEEE Transactions on Image Processing, 26(7):3142–3155, 2017.
 [37] Kai Zhang, Wangmeng Zuo, and Lei Zhang. Ffdnet: Toward a fast and flexible solution for cnnbased image denoising. IEEE Transactions on Image Processing, 27(9):4608–4622, 2018.
 [38] Xiaoqin Zhang, Di Wang, Zhengyuan Zhou, and Yi Ma. Robust lowrank tensor recovery with rectification and alignment. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019.
 [39] Yongqin Zhang, Jinsheng Xiao, Jinye Peng, Yu Ding, Jiaying Liu, Zongming Guo, and Xiaopeng Zong. Kernel wiener filtering model with lowrank approximation for image denoising. Information Sciences, 462:402–416, 2018.
 [40] Daniel Zoran and Yair Weiss. From learning models of natural image patches to whole image restoration. In IEEE International Conference on Computer Vision (ICCV), pages 479–486, 2011.