Log In Sign Up

Low rank prior and l0 norm to remove impulse noise in images

by   Haijuan Hu, et al.

Patch-based low rank is an important prior assumption for image processing. Moreover, according to our calculation, the optimization of l0 norm corresponds to the maximum likelihood estimation under random-valued impulse noise. In this article, we thus combine exact rank and l0 norm for removing the noise. It is solved formally using the alternating direction method of multipliers (ADMM), with our previous patch-based weighted filter (PWMF) producing initial images. Since this model is not convex, we consider it as a Plug-and-Play ADMM, and do not discuss theoretical convergence properties. Experiments show that this method has very good performance, especially for weak or medium contrast images.


page 4

page 6

page 7

page 8

page 9


Sparse + Low Rank Decomposition of Annihilating Filter-based Hankel Matrix for Impulse Noise Removal

Recently, so called annihilating filer-based low rank Hankel matrix (ALO...

Patch-Based Low-Rank Minimization for Image Denoising

Patch-based sparse representation and low-rank approximation for image p...

Occluded Face Recognition Using Low-rank Regression with Generalized Gradient Direction

In this paper, a very effective method to solve the contiguous face occl...

Sparse and Low-Rank Covariance Matrices Estimation

This paper aims at achieving a simultaneously sparse and low-rank estima...

1-Bit Matrix Completion under Exact Low-Rank Constraint

We consider the problem of noisy 1-bit matrix completion under an exact ...

Sparse Trace Norm Regularization

We study the problem of estimating multiple predictive functions from a ...

Bilinear Parameterization For Differentiable Rank-Regularization

Low rank approximation is a commonly occurring problem in many computer ...

1 Introduction

Image restoration is a fundamental problem in image processing. We focus on here random-valued impulse noise, often added during image acquisition and transmission. The task is to recover original image from the noisy one by removing noise.

To recover images, one should assume images have some essential properties. Traditional prior assumptions are smoothnesshunt1973application , piecewise constantrudin_nonlinear_1992 , etc. Among many priors developed later, two important priors are patch similaritybuades_review_2006 , and low rank property gu2014weighted ; hu2018note . Patch similarity assumes that there exist many similar patches in images. Low rank property means the matrix composed of similar patches has low rank. Based on patch similarity and a statistics ROADgarnett2005universal for detecting impulse noise, Hu et alhu2016removing propose patch-based weighted filter (PWMF) for removing impulse noise and mixed noise. Since the optimization of the rank is generally a NP-hard problem, many researches use the nuclear norm or the weighted nuclear norm to approximate the rank, such as WNNMgu2014weighted for removing Gaussian noise. But in some cases, the rank optimization has a close-form solution: Hu et alhu2018note propose an exact low rank model for removing Gaussian noise. In jin2018sparse , the authors use the sparse and low rank decomposition of a Hankel structured matrix for impulse noise removal, where norm is used for the sparseness, and nuclear norm for approximating rank. In adam2021combined , the authors propose a variational model that uses the nonconvex -norm,

for both the data fidelity and a regularization term, combined with a second-order total variation regularization term and an overlapping group sparse regularizer. Recently deep learning is also applied for removing impulse noise, and has reached very good performance

jin2019learning ; wang2020variational .

However, to remove impulse noise, the exact low rank properties are not exploited in the literature as far as we know. In this paper, we will extend the exact low rank modelhu2018note to remove impulse noise. Moreover, according to our computation, norm fidelity corresponds to the maximum likelihood estimate under impulse noise, noting that norm corresponds to Gaussian noise. Thus we optimize a combination of exact rank and norm fidelity, which is solved formally by the alternating direction method of multipliers (ADMM) goldstein2009split . We consider the method as a Plug-and-Play ADMMvenkatakrishnan2013plug , and do not discuss theoretical convergence properties. Since the model is not convex, the results depend on the initial image. We choose the PWMF results as initial images, which can also simplify the model. Experiments show that this algorithm produces very good images, especially for low or medium contrast images.

2 Patch-based low rank method

In hu2018note , Hu et al propose a patch-based low rank method (PLR) for removing Gaussian noisse. Firstly, the noisy image is divided into overlapping patches of size . For each patch, its most similar patches located in its

neighbor region are grouped together to form a similarity matrix, each column being a vectorization of a similar patch. The similarity is determined by Euclidean norm. For each similarity matrix

, it is denoised by the exact low rank model


where the minimum is taken over all the matrices having the same size as , is the Frobenius norm, and represents a threshold parameter(). The solution can be obtained by


where , , and are derived from the SVD of , that is, , and is the hard thresholding operator:


Finally, each denoised similarity matrix is returned to the original location; the overlapping denoised values for each pixel are aggregated. The parameters are chosen as .

3 Remove impulse noise

The noise model is as follows,


where is the original image, is the observed noisy one,

is independent random noise uniformly distributed on [min

, max, generally taken as [0,255] for 8-bit gray images, and is the proportion of noise, representing the noise level.

As it is known that the fidelity term for Gaussian noise is norm, to obtain the fidelity term for impulse noise, we consider the maximum likelihood estimation (MLE) for . For simplicity, we utilize the discrete model. That is, for two pixels and of the same location of and respectively,


Given independent realizations of , the likelihood function is then


where is the number taking the value . Recall that


Since , is an increasing function of . That is,


which means the fidelity term for impulse noise is norm.

According to above analysis, the following model is considered to remove impulse noise,


where denotes similarity matrices. Since the model is not convex, it is solved formally by ADMM goldstein2009split ; ma2017low .


We use PWMF to produce initial image for the iteration, since it runs fast and has good results; the initial image

is the zero matrix. By an element-wise calculation, the minimizer of (

10) has a close form:


The -subproblem (11) is solved as PLR in Section 2, with regarded as Gaussian noisy images and . Since contains little impulse noise, are good estimations of original images, which are thus used directly for the selection of similar patches for -sub problem as PLR.

4 Experiments

To show the performance of the proposed method, we test some classic images, the original clean images of which are displayed in Figure 1

. We use peak signal-to-noise ratio (PSNR) values to measure the quality of denoised images

quantitatively, which are defined by PSNRdB with being the squared root of the number of elements of the image . For our method, we only need to choose the values of and , and determine the stop criteria for the iteration. We use PLR with in (1) for the solution of (11) assuming has Gaussian noise level . We choose empirically by obtaining the highest average PSNR values of tested images, which is taken as for all the impulse noise levels . Comparing (11) and (1), we thus use . In addition, the fixed iteration times 50 is used.

The proposed method is compared with some recently published methods, robust ALOHAjin2018sparse , HNHOTV-OGSadam2021combined , and our previous method PWMFhu2016removing . We consider four noise level with , and 0.5. For PWMF, we use the same parameters as in the paperhu2016removing . For robust ALOHA, since the paperjin2018sparse only considers and , we use the same parameters as when and 0.3, and the same parameters as when and 0.5. For HNHOTV-OGSadam2021combined , according to the paper, we adjust the parameters in [43,79], and in [4.8, 7.5],which are finally chosen as for ; for ; for , and for by obtaining highest average PSNR values of tested images.

The PSNR values of different methods are shown in Table 1. It is shown that our method is better than other methods for most of the images, especially for images with weak contrast, such as Lena and House. Robust ALOHAjin2018sparse is especially good for the high contrast image Barbara and is better for the image Cameraman when than other methods, but it is not as good as HNHOTV-OGSadam2021combined or the proposed method in other cases. HNHOTV-OGS is better than the proposed method in only three cases, and the differences are small. The proposed method improves PWMF in all cases. In Figures 2,3,4, and 5, some images are displayed to show the differences of the methods. From these figures, it can be seen that our method improves PWMF: it retains clear image details in textural parts and edges, and also recovers well homogeneous regions. From Figures 2 and 4, it can be observed that robust ALOHA loses more details than our methods, while some noise or artifacts appear in Figures 3 and 5. The image details obtained by HNHOTV-OGS are not as clear as the proposed method either, which can be seen in Figures 3, 4, and 5.

Note that we use the same parameters for all the tested images. Since different images have different structures, the denoising performance can be improved with adaptive parameters, especially for high contrast images, which will be our future work.

Bar Boa Cam Cou Hill Hou Lena Man Mon Pep
28.43 31.86 27.26 31.94 33.59 35.06 35.77 33.07 28.63 32.08
34.79 31.05 28.09 31.06 32.65 34.57 35.36 31.15 28.29 30.10
25.83 32.49 27.13 33.36 33.84 37.24 36.48 32.86 30.29 31.87
29.46 33.68 27.68 33.51 36.26 40.37 38.83 34.11 29.97 33.17
26.33 29.76 25.67 29.69 31.16 32.70 33.61 30.88 26.93 29.84
31.97 29.59 24.42 28.49 29.21 33.05 32.74 29.63 26.23 28.19
24.62 30.31 25.45 30.74 31.28 33.37 33.69 30.87 27.56 29.87
27.1 31.33 25.95 31.20 33.73 37.08 36.38 32.03 28.10 30.84
25.09 27.77 23.74 27.65 29.83 30.93 31.81 29.31 25.06 28.03
29.30 26.84 23.53 26.91 28.59 30.27 30.11 27.24 23.25 26.24
23.60 28.28 23.88 28.69 29.19 30.57 31.39 29.12 25.08 28.16

28.59 23.76 28.62 32.07 33.75 33.46 29.91 25.62 28.66
24.06 26.57 23.01 26.38 28.52 29.40 30.38 27.99 23.67 26.36
25.61 24.21 20.73 24.63 25.36 26.88 27.70 25.65 21.3 23.48
23.00 26.34 22.66 26.48 27.26 28.77 29.50 27.32 22.99 26.34
24.53 27.07 23.04 27.08 30.12 31.57 31.75 28.42 24.15 26.94
Table 1: PSNR values for removing impulse noise with and 0.5. For each , from top to bottom, the methods are PWMFhu2016removing , robust ALOHAjin2018sparse , HNHOTV-OGSadam2021combined , and the proposed method. From left to right, the images are Barbara, Boats, Cameraman, Couple, Hill, House, Lena, Man, Monarch, Peppers.
Barbara Boats Cameraman Couple Hill
House Lena Man Monarch Peppers

Figure 1: Original images tested
Original Noisy PWMF
Robust ALOHA HNHOTV-OGS Proposed

Figure 2: Denoised images for . The third and bottom rows show the enlarged versions of the same part of the Peppers images from the first and second rows respectively.
Original Noisy PWMF
Robust ALOHA HNHOTV-OGS Proposed

Figure 3: Denoised images for The third and bottom rows show the enlarged versions of the same part of the House images from the first and second rows respectively.
Original Noisy PWMF
Robust ALOHA HNHOTV-OGS Proposed

Figure 4: Denoised images for . The third and bottom rows show the enlarged versions of the same part of the Lena images from the first and second rows respectively.
Original Noisy PWMF
Robust ALOHA HNHOTV-OGS Proposed

Figure 5: Denoised images for . The third and bottom rows show the enlarged versions of the same part of the Boats images from the first and second rows respectively.



  • (1) B. Hunt, The application of constrained least squares estimation to image restoration by digital computer, IEEE Transactions on Computers, 100 (9) (1973) 805–812.
  • (2) L. I. Rudin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena 60 (1-4) (1992) 259–268.
  • (3) A. Buades, B. Coll, J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul. 4 (2) (2006) 490–530.
  • (4)

    S. Gu, L. Zhang, W. Zuo, X. Feng, Weighted nuclear norm minimization with application to image denoising, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 2862–2869.

  • (5) H. Hu, J. Froment, Q. Liu, A note on patch-based low-rank minimization for fast image denoising, J. Visual Commun. Image Representation 50 (2018) 100–110.
  • (6) R. Garnett, T. Huegerich, C. Chui, W. He, A universal noise removal algorithm with an impulse detector, IEEE Trans. Image Process. 14 (11) (2005) 1747–1754.
  • (7) H. Hu, B. Li, Q. Liu, Removing mixture of gaussian and impulse noise by patch-based weighted means, J. Sci. Comput. 67 (1) (2016) 103–129.
  • (8) K. H. Jin, J. C. Ye, Sparse and low-rank decomposition of a hankel structured matrix for impulse noise removal, IEEE Trans. Image Process. 27 (2018) 1448–1461.
  • (9) T. Adam, R. Paramesran, M. Yin, K. Ratnavelu, Combined higher order non-convex total variation with overlapping group sparsity for impulse noise removal, Multim. Tools Appl. 80 (2021) 18503–18530.
  • (10) L. Jin, W. Zhang, G. Ma, E. Song, Learning deep cnns for impulse noise removal in images, J. Vis. Commun. Image Represent. 62 (2019) 193–205.
  • (11) F. Wang, H. Huang, J. Liu, Variational-based mixed noise removal with cnn deep learning regularization, IEEE Trans. Image Process. 29 (2020) 1246–1258.
  • (12) T. Goldstein, S. Osher, The split bregman method for l1-regularized problems, SIAM J. Imaging Sci. 2 (2) (2009) 323–343.
  • (13) S. Venkatakrishnan, C. A. Bouman, B. Wohlberg, Plug-and-play priors for model based reconstruction, 2013 IEEE Global Conference on Signal and Information Processing (2013) 945–948.
  • (14) L. Ma, L. Xu, T. Zeng, Low rank prior and total variation regularization for image deblurring, J. Sci. Comput. 70 (2017) 1336–1357.