1 Introduction
Image restoration is a fundamental problem in image processing. We focus on here random-valued impulse noise, often added during image acquisition and transmission. The task is to recover original image from the noisy one by removing noise.
To recover images, one should assume images have some essential properties. Traditional prior assumptions are smoothnesshunt1973application , piecewise constantrudin_nonlinear_1992 , etc. Among many priors developed later, two important priors are patch similaritybuades_review_2006 , and low rank property gu2014weighted ; hu2018note . Patch similarity assumes that there exist many similar patches in images. Low rank property means the matrix composed of similar patches has low rank. Based on patch similarity and a statistics ROADgarnett2005universal for detecting impulse noise, Hu et alhu2016removing propose patch-based weighted filter (PWMF) for removing impulse noise and mixed noise. Since the optimization of the rank is generally a NP-hard problem, many researches use the nuclear norm or the weighted nuclear norm to approximate the rank, such as WNNMgu2014weighted for removing Gaussian noise. But in some cases, the rank optimization has a close-form solution: Hu et alhu2018note propose an exact low rank model for removing Gaussian noise. In jin2018sparse , the authors use the sparse and low rank decomposition of a Hankel structured matrix for impulse noise removal, where norm is used for the sparseness, and nuclear norm for approximating rank. In adam2021combined , the authors propose a variational model that uses the nonconvex -norm,
for both the data fidelity and a regularization term, combined with a second-order total variation regularization term and an overlapping group sparse regularizer. Recently deep learning is also applied for removing impulse noise, and has reached very good performance
jin2019learning ; wang2020variational .However, to remove impulse noise, the exact low rank properties are not exploited in the literature as far as we know. In this paper, we will extend the exact low rank modelhu2018note to remove impulse noise. Moreover, according to our computation, norm fidelity corresponds to the maximum likelihood estimate under impulse noise, noting that norm corresponds to Gaussian noise. Thus we optimize a combination of exact rank and norm fidelity, which is solved formally by the alternating direction method of multipliers (ADMM) goldstein2009split . We consider the method as a Plug-and-Play ADMMvenkatakrishnan2013plug , and do not discuss theoretical convergence properties. Since the model is not convex, the results depend on the initial image. We choose the PWMF results as initial images, which can also simplify the model. Experiments show that this algorithm produces very good images, especially for low or medium contrast images.
2 Patch-based low rank method
In hu2018note , Hu et al propose a patch-based low rank method (PLR) for removing Gaussian noisse. Firstly, the noisy image is divided into overlapping patches of size . For each patch, its most similar patches located in its
neighbor region are grouped together to form a similarity matrix, each column being a vectorization of a similar patch. The similarity is determined by Euclidean norm. For each similarity matrix
, it is denoised by the exact low rank model(1) |
where the minimum is taken over all the matrices having the same size as , is the Frobenius norm, and represents a threshold parameter(). The solution can be obtained by
(2) |
where , , and are derived from the SVD of , that is, , and is the hard thresholding operator:
(3) |
Finally, each denoised similarity matrix is returned to the original location; the overlapping denoised values for each pixel are aggregated. The parameters are chosen as .
3 Remove impulse noise
The noise model is as follows,
(4) |
where is the original image, is the observed noisy one,
is independent random noise uniformly distributed on [min
, max, generally taken as [0,255] for 8-bit gray images, and is the proportion of noise, representing the noise level.As it is known that the fidelity term for Gaussian noise is norm, to obtain the fidelity term for impulse noise, we consider the maximum likelihood estimation (MLE) for . For simplicity, we utilize the discrete model. That is, for two pixels and of the same location of and respectively,
(5) |
Given independent realizations of , the likelihood function is then
(6) |
where is the number taking the value . Recall that
(7) |
Since , is an increasing function of . That is,
(8) |
which means the fidelity term for impulse noise is norm.
According to above analysis, the following model is considered to remove impulse noise,
(9) |
where denotes similarity matrices. Since the model is not convex, it is solved formally by ADMM goldstein2009split ; ma2017low .
(10) | |||||
(11) | |||||
(12) |
We use PWMF to produce initial image for the iteration, since it runs fast and has good results; the initial image
is the zero matrix. By an element-wise calculation, the minimizer of (
10) has a close form:(13) |
The -subproblem (11) is solved as PLR in Section 2, with regarded as Gaussian noisy images and . Since contains little impulse noise, are good estimations of original images, which are thus used directly for the selection of similar patches for -sub problem as PLR.
4 Experiments
To show the performance of the proposed method, we test some classic images, the original clean images of which are displayed in Figure 1
. We use peak signal-to-noise ratio (PSNR) values to measure the quality of denoised images
quantitatively, which are defined by PSNRdB with being the squared root of the number of elements of the image . For our method, we only need to choose the values of and , and determine the stop criteria for the iteration. We use PLR with in (1) for the solution of (11) assuming has Gaussian noise level . We choose empirically by obtaining the highest average PSNR values of tested images, which is taken as for all the impulse noise levels . Comparing (11) and (1), we thus use . In addition, the fixed iteration times 50 is used.The proposed method is compared with some recently published methods, robust ALOHAjin2018sparse , HNHOTV-OGSadam2021combined , and our previous method PWMFhu2016removing . We consider four noise level with , and 0.5. For PWMF, we use the same parameters as in the paperhu2016removing . For robust ALOHA, since the paperjin2018sparse only considers and , we use the same parameters as when and 0.3, and the same parameters as when and 0.5. For HNHOTV-OGSadam2021combined , according to the paper, we adjust the parameters in [43,79], and in [4.8, 7.5],which are finally chosen as for ; for ; for , and for by obtaining highest average PSNR values of tested images.
The PSNR values of different methods are shown in Table 1. It is shown that our method is better than other methods for most of the images, especially for images with weak contrast, such as Lena and House. Robust ALOHAjin2018sparse is especially good for the high contrast image Barbara and is better for the image Cameraman when than other methods, but it is not as good as HNHOTV-OGSadam2021combined or the proposed method in other cases. HNHOTV-OGS is better than the proposed method in only three cases, and the differences are small. The proposed method improves PWMF in all cases. In Figures 2,3,4, and 5, some images are displayed to show the differences of the methods. From these figures, it can be seen that our method improves PWMF: it retains clear image details in textural parts and edges, and also recovers well homogeneous regions. From Figures 2 and 4, it can be observed that robust ALOHA loses more details than our methods, while some noise or artifacts appear in Figures 3 and 5. The image details obtained by HNHOTV-OGS are not as clear as the proposed method either, which can be seen in Figures 3, 4, and 5.
Note that we use the same parameters for all the tested images. Since different images have different structures, the denoising performance can be improved with adaptive parameters, especially for high contrast images, which will be our future work.
Bar | Boa | Cam | Cou | Hill | Hou | Lena | Man | Mon | Pep |
---|---|---|---|---|---|---|---|---|---|
28.43 | 31.86 | 27.26 | 31.94 | 33.59 | 35.06 | 35.77 | 33.07 | 28.63 | 32.08 |
34.79 | 31.05 | 28.09 | 31.06 | 32.65 | 34.57 | 35.36 | 31.15 | 28.29 | 30.10 |
25.83 | 32.49 | 27.13 | 33.36 | 33.84 | 37.24 | 36.48 | 32.86 | 30.29 | 31.87 |
29.46 | 33.68 | 27.68 | 33.51 | 36.26 | 40.37 | 38.83 | 34.11 | 29.97 | 33.17 |
26.33 | 29.76 | 25.67 | 29.69 | 31.16 | 32.70 | 33.61 | 30.88 | 26.93 | 29.84 |
31.97 | 29.59 | 24.42 | 28.49 | 29.21 | 33.05 | 32.74 | 29.63 | 26.23 | 28.19 |
24.62 | 30.31 | 25.45 | 30.74 | 31.28 | 33.37 | 33.69 | 30.87 | 27.56 | 29.87 |
27.1 | 31.33 | 25.95 | 31.20 | 33.73 | 37.08 | 36.38 | 32.03 | 28.10 | 30.84 |
25.09 | 27.77 | 23.74 | 27.65 | 29.83 | 30.93 | 31.81 | 29.31 | 25.06 | 28.03 |
29.30 | 26.84 | 23.53 | 26.91 | 28.59 | 30.27 | 30.11 | 27.24 | 23.25 | 26.24 |
23.60 | 28.28 | 23.88 | 28.69 | 29.19 | 30.57 | 31.39 | 29.12 | 25.08 | 28.16 |
25.64 |
28.59 | 23.76 | 28.62 | 32.07 | 33.75 | 33.46 | 29.91 | 25.62 | 28.66 |
24.06 | 26.57 | 23.01 | 26.38 | 28.52 | 29.40 | 30.38 | 27.99 | 23.67 | 26.36 |
25.61 | 24.21 | 20.73 | 24.63 | 25.36 | 26.88 | 27.70 | 25.65 | 21.3 | 23.48 |
23.00 | 26.34 | 22.66 | 26.48 | 27.26 | 28.77 | 29.50 | 27.32 | 22.99 | 26.34 |
24.53 | 27.07 | 23.04 | 27.08 | 30.12 | 31.57 | 31.75 | 28.42 | 24.15 | 26.94 |
![]() |
![]() |
![]() |
![]() |
![]() |
Barbara | Boats | Cameraman | Couple | Hill |
![]() |
![]() |
![]() |
![]() |
![]() |
House | Lena | Man | Monarch | Peppers |
![]() |
![]() |
![]() |
Original | Noisy | PWMF |
![]() |
![]() |
![]() |
Robust ALOHA | HNHOTV-OGS | Proposed |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Original | Noisy | PWMF |
![]() |
![]() |
![]() |
Robust ALOHA | HNHOTV-OGS | Proposed |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|
![]() |
![]() |
![]() |
Original | Noisy | PWMF |
![]() |
![]() |
![]() |
Robust ALOHA | HNHOTV-OGS | Proposed |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
Original | Noisy | PWMF |
![]() |
![]() |
![]() |
Robust ALOHA | HNHOTV-OGS | Proposed |
![]() |
![]() |
![]() |
![]() |
![]() |
![]() |
|
References
References
- (1) B. Hunt, The application of constrained least squares estimation to image restoration by digital computer, IEEE Transactions on Computers, 100 (9) (1973) 805–812.
- (2) L. I. Rudin, S. Osher, E. Fatemi, Nonlinear total variation based noise removal algorithms, Physica D: Nonlinear Phenomena 60 (1-4) (1992) 259–268.
- (3) A. Buades, B. Coll, J. M. Morel, A review of image denoising algorithms, with a new one, Multiscale Model. Simul. 4 (2) (2006) 490–530.
-
(4)
S. Gu, L. Zhang, W. Zuo, X. Feng, Weighted nuclear norm minimization with application to image denoising, in: Proc. IEEE Conf. Comput. Vis. Pattern Recognit., 2014, pp. 2862–2869.
- (5) H. Hu, J. Froment, Q. Liu, A note on patch-based low-rank minimization for fast image denoising, J. Visual Commun. Image Representation 50 (2018) 100–110.
- (6) R. Garnett, T. Huegerich, C. Chui, W. He, A universal noise removal algorithm with an impulse detector, IEEE Trans. Image Process. 14 (11) (2005) 1747–1754.
- (7) H. Hu, B. Li, Q. Liu, Removing mixture of gaussian and impulse noise by patch-based weighted means, J. Sci. Comput. 67 (1) (2016) 103–129.
- (8) K. H. Jin, J. C. Ye, Sparse and low-rank decomposition of a hankel structured matrix for impulse noise removal, IEEE Trans. Image Process. 27 (2018) 1448–1461.
- (9) T. Adam, R. Paramesran, M. Yin, K. Ratnavelu, Combined higher order non-convex total variation with overlapping group sparsity for impulse noise removal, Multim. Tools Appl. 80 (2021) 18503–18530.
- (10) L. Jin, W. Zhang, G. Ma, E. Song, Learning deep cnns for impulse noise removal in images, J. Vis. Commun. Image Represent. 62 (2019) 193–205.
- (11) F. Wang, H. Huang, J. Liu, Variational-based mixed noise removal with cnn deep learning regularization, IEEE Trans. Image Process. 29 (2020) 1246–1258.
- (12) T. Goldstein, S. Osher, The split bregman method for l1-regularized problems, SIAM J. Imaging Sci. 2 (2) (2009) 323–343.
- (13) S. Venkatakrishnan, C. A. Bouman, B. Wohlberg, Plug-and-play priors for model based reconstruction, 2013 IEEE Global Conference on Signal and Information Processing (2013) 945–948.
- (14) L. Ma, L. Xu, T. Zeng, Low rank prior and total variation regularization for image deblurring, J. Sci. Comput. 70 (2017) 1336–1357.