NAMF: A Non-local Adaptive Mean Filter for Salt-and-Pepper Noise Removal

10/17/2019 ∙ by Houwang Zhang, et al. ∙ 15

In this paper, a non-local adaptive mean filter (NAMF) is proposed, which can eliminate all levels of salt-and-pepper (SAP) noise. NAMF can be divided into two stages: (1) SAP noise detection; (2) SAP noise elimination. For a given pixel, firstly, we compare it with the maximum or minimum gray value of the noisy image, if it equals then we use a window with adaptive size to further determine whether it is noisy, and the noiseless pixel will be left. Secondly, the noisy pixel will be replaced by the combination of its neighboring pixels. And finally we use a SAP noise based non-local mean filter to further restore it. Our experimental results show that NAMF outperforms state-of-the-art methods in terms of quality for restoring image at all SAP noise levels.



There are no comments yet.


page 1

page 2

page 3

page 4

page 5

Code Repositories



view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Digital images are often corrupted by impulse noise for a variety of causes in the process of image acquisition and transmission [1, 2]. As a preprocessing step in image processing, image denoising can protect edges, textures and other details, hence it becomes one of the most important tasks in image processing [3]. Salt-and-pepper (SAP) noise exists widely in natural images, the pixels contaminated by SAP noise take the maximum or minimum value and show as black or white points [4].

In order to remove SAP noise, lots of effective methods have been proposed. Among them, median filter (MF) and adaptive median filter (AMF) [5] are two most widely used methods in the early stage. MF can restore image details well under low noise intensity, but it performs poorly when noise intensity is high [6]. AMF adopts the measure of window with adaptive size, which makes it perform well in high noise intensity [7].

In recent years, studies on denoising of SAP noise are mainly based on MF and AMF. Based on AMF, NAFSMF recognizes SAP noise by analyzing the histogram of noisy images and takes a fuzzy method to denoise [8]. AWMF uses two successive windows to detect noisy pixels and processes them with a weighted mean filter [9]. The method proposed in [10] uses a new adaptive fuzzy switching weighted mean filter to remove SAP noise, which is based on NAFSMF and AWMF. BPDF removes SAP noise through searching the repeat numbers of the pixels, and achieves a good performance under low SAP noise intensity [11]. DAMF uses a different applied median filter to remove SAP noise [12].

The restoration for SAP noise is just using the rest information to repair the absent information. The problem of exiting state-of-the-art methods is that under high SAP noise level, the boundary of the restored image is jagged, the details are blurred, as shown in Fig. 1. (b). When SAP noise intensity is too high, the lost information of the image is too much. The consequence is that less available information can be used to restore the image.

Figure 1: (a) “Lena” corrupted by SAP noise ratio of 10%, and the gray values of noisy candidates in and are both . (b) Result of the method proposed in [10] for “Lena” corrupted by SAP noise ratio of 90%.

To solve the above problem, in this paper, we propose a non-local adaptive mean filter (NAMF), which can remove SAP noise at all levels. NAMF can be divided into two stages: (1) SAP noise detection; (2) SAP noise elimination. Firstly, in the stage of SAP noise detection, compare whether each pixel is equal to the global maximum or minimum gray value of the image. If it is then we take it as a noisy candidate and use a window with adaptive size to confirm it. We calculate the proportion of the pixels with the same value of candidate in the window, and then compare it with a threshold. If smaller, the candidate is regarded as a noisy pixel, otherwise it is noiseless and will not be processed. Secondly, in the stage of SAP noise elimination, the noisy pixel will be replaced by the mean of its neighboring pixels. And then we use a SAP noise based non-local mean filter to further restore it. The main contributions of this paper can be concluded as follows:

  • We propose a proportion based method to distinguish the noise pixels and texture pixels.

  • We propose an improved non-local mean method based on the characteristics of SAP noise to further restore noisy image.

Experiment results demonstrate that NAMF outperforms the exiting sate-of-the-art methods under both high SAP noise level and low SAP noise level.

2 Non-local adaptive mean filter

2.1 SAP Noise Detection

Let denote an original 8-bit gray-level image with size of and denote a noisy image of corrupted by SAP noise. and represent the gray value of the pixel at location of and , respectively, where .

For , the values of each pixel is in the range of . For , its noisy pixels have the maximum value or minimum value , so the pixel is defined as Eq. 1.


where is the probability of noisy pixels, also the density of SAP noise of . And in the process of image denoising, we use to represent a window centered at .

Considering the characteristics of SAP noise, pixel corrupted by SAP noise is or , that is to say, noisy pixel candidate only has 2 possible values: and . Here we introduce a prior decision condition in noise detection:


where is the value of pixel in the tag matrix , means pixel is the noisy pixel candidate, while means pixel is noiseless. For a natural image, the pixels with high or low value are also possible to be the texture of the image, hence it is very necessary to further confirm these pixels with maximum or minimum value.

Unlike SAP noise, texture pixels are usually continuous and non-discrete. So we use the proportion of similar pixels in an adaptive size window centered at the candidate to further determine whether the candidate is noisy or not. For example, as shown in Fig. 1. (a), and are both candidates with gray value of (black), but has a larger possibility to be a noisy pixel.

In the detection of SAP noise, size of the window is initialed to 1. If the condition is met ( or ), the is just we need. If not, then and continue to compute, where is the number of pixels within which are not equal to and , and is the maximum size of window. After the computation of , if , we take the pixel in the window as a noisy pixel. If not, it may be a texture pixel, then we compare the proportion with the threshold , where is the number of pixels owning same value as candidate pixel in the window . If , the candidate pixel is regarded as a noisy pixel, else it is noiseless.

For pixel , if it is detected as noisy, we mark it with the discriminant matrix (), and , else and will not be processed.

2.2 SAP Noise Elimination

We restore noisy pixels in two steps. Let represent the initially restored image, represent the final output image. Before processing, we initialize .

Firstly, when a pixel is detected as a noisy pixel, we use to restore it. The calculation of is based on as shown in Eq. 3. When , is the mean of the noiseless pixels in , otherwise is the mean of three processed neighboring pixels of in . Unlike the four neighbors adopted in [10], the utilization of four neighboring pixels will lead to residual noisy pixels on the boundary, as shown in Fig. 1. (b) (even if the image boundary is expanded during process, some noisy pixels of boundary can not be restored). Hence, we select three processed neighboring pixels in , i.e. , and its illustration is shown in Fig. 2.


Figure 2: Three neighboring pixels and selected in for restoring pixel in when .

Secondly, considering the problem that restoration will be hard under high noise intensity, we introduce the non-local mean (NLM) method [13] to further restore noisy pixels. NLM can restore noisy pixels by using all neighbors instead of part of neighbors which are detected as noiseless, in this way we can further enhance the restoration effect. And original NLM is designed for Gaussian noise, we modify it based on the characteristics of SAP noise to apply it in our method.

In the noise detection stage, pixel is identified and marked by the discriminant matrix . means that the pixel is identified as a noisy pixel, otherwise noiseless (here for convenience we ignore the subscript, is written as , is , is ). After the first step for the restoration of the noisy pixel , we get . The modified NLM method is used to further restore . We use to represent the value of the further restored , its calculation is as follows:


where represents a searching window of size centered at , and represents the weight of pixel in . , also called similarity window, is a square block centered at , so is . As shown in Eq. 6, the similarity between and is measured by the Gaussian weighted Euclidean distance between and , where

is the standard deviation of the Gaussian kernel. And

is the smoothing parameter for NLM.

When processing the noisy pixel , original NLM assigns the weight according to the similarity, that is to say, the weight of pixel itself is the largest. Different from original NLM, in our method, the noisy pixel to be processed will not participate in the process of NLM, so the weight of the pixel should be set as , as shown in Eq. 5.

In the NLM algorithm, the higher noise intensity is, the larger smoothing parameter should be. But intensity of noise is not easy to be confirmed. Considering that SAP noise can be significantly detected, we use the intensity of SAP noise to confirm . As shown in Eq. 6, represents the total number of non-zero elements in the discriminant matrix , that is, the more noisy pixels are detected, the larger should be, and are the parameters used to fit .

Considering the high computational cost of NLM algorithm, we introduce a kind of fast implementation of NLM algorithm [14] based on the computation of patch distances using sums of lines to accelerate our NAMF algorithm. The details of the proposed NAMF are shown in Algorithm 1.

/*STAGE 1*/
 Compute tag matrix , initialize .
 For each pixel in the noisy image and the
 initially restored image , do
1) If , break; Otherwise go to step 2).
2) Initialize , , .
3) Compute until or ;
 Otherwise, and repeat step 3).
4) If , , ,
 break; Otherwise, go to step 5).
5) Compute . If , , ;
 Otherwise, .
/*STAGE 2*/
 Compute . And for each pixel in image and output
 image , do
6) If , ;
 Otherwise, .
Algorithm 1 NAMF

3 Experimental Results

Figure 3: Experimental results of different methods for “Barbara” with SAP noise ratio of 10% and “Lena” with SAP noise ratio of 90%. (a) Noisy image. (b) AMF. (c) NAFSMF. (d) AWMF. (e) [10]. (f) BPDF. (g) DAMF. (h) NAMF. (i) Original image.

In the experiments, the results of NAMF are compared with six state-of-the-art methods: AMF [5], NAFSMF [8], AWMF [9], the method proposed in [10], BPDF [11], DAMF [12]. Sixteen typical images are chosen for the experiments including eight images (“Barbara”, “Elaine”, “Goldhill”, “Lena”, “Man”, “Peppers”, “Yacht”, and “Zelda”) and eight

images (“Baboon”, “House”, “Boat”, “Cameraman”, “Einstein”, “Face”, “Straw”, and “Couple”). We use two typical image quality metrics, peak signal-to-noise ratio (PSNR)

[15] and structural similarity (SSIM) [16] to evaluate the restoring quality, respectively. The experiments are performed on a personal computer with Windows 10, Intel(R) Core(TM) i7 2.2 GHz 6 cores CPU, and 8 GB RAM.

In this paper, we set , and to fit , the size of searching window is , the size of similarity window is . Through test, we finally take threshold for our method. Other methods keep the default parameters.

Fig. 3 shows the restored results for “Barbara” with SAP noise ratio of 10% and “Lena” with SAP noise ratio of 90%, respectively. By observing Fig. 3, it can be found that NAMF can restore more image details. And when noise intensity is high, restored images by other methods are very blurred, while result of our method looks more natural.

Figure 4: Average PSNR of different methods at all SAP noise levels.

Figure 5: Average SSIM of different methods at all SAP noise levels.

The curves of average PSNR and SSIM are shown in Fig. 4 and Fig. 5, respectively. By observing Fig. 4, we can find that NAMF obtains the highest PSNR under both low noise intensity and high noise intensity, and PSNR of our method is much higher than results of other methods. Fig. 5 shows that the SSIM curves of most methods are basically the same under low SAP noise intensity. However, with the increasing of noise level, the superiority of NAMF becomes more and more obviously, after the noise ratio exceeds 30%, the SSIM obtained by NAMF is significantly higher than other methods.

Fig. 6 shows the average running time of different methods at all noise levels. With the increasing of noise intensity, the running time of all methods except AWMF increases. Clearly, the average running time of the method proposed in [10] is the shortest. Although the rank of NAMF is in the middle, its processing speed is superior against NAFSMF and similar to DAMF under high noise intensity.

4 Conclusion

Figure 6: Average running time of different methods at all SAP noise levels.

In this paper, we have proposed a method called NAMF for SAP noise denoising, which adopts a SAP noise based non-local mean method. NAMF can get much higher restoring quality than state-of-the-art methods at all SAP noise levels. The processing time of NAMF is comparable to most state-of-the-art methods. Experimental results show that NAMF can get much better PSNR and SSIM at all SAP noise levels, that is to say, the image restored by NAMF is more natural under both high or low noise intensity.


  • [1] Vivek Chandra, Sagar Deokar, Siddhant Badhe, and Rajesh Yawle, “Removal of high density salt and pepper noise through modified decision based unsymmetric trimmed adaptive median filter,” IEEE Signal Processing Letters, vol. 18, no. 5, pp. 287–290, 2011.
  • [2] Wenbin Luo, “Efficient removal of impulse noise from digital images,” Consumer Electronics IEEE Transactions on, vol. 52, no. 2, pp. 523–527, 2006.
  • [3] Shaoping Xu, Xiaohui Yang, and Shunliang Jiang, “A fast nonlocally centralized sparse representation algorithm for image denoising,” Signal Processing, vol. 131, pp. 99–112, 2017.
  • [4] Vikas Singh, Raghav Dev, Narendra K Dhar, Pooja Agrawal, and Nishchal K Verma, “Adaptive type-2 fuzzy approach for filtering salt and pepper noise in grayscale images,” IEEE transactions on fuzzy systems, vol. 26, no. 5, pp. 3170–3176, 2018.
  • [5] H. Hwang and R. A. Haddad, “Adaptive median filters: new algorithms and results,” IEEE Transactions on Image Processing, vol. 4, no. 4, pp. 499–502, 1995.
  • [6] Raymond H Chan, Ho Chung-Wa, and Nikolova Mila, “Salt-and-pepper noise removal by median-type noise detectors and detail-preserving regularization,” IEEE Transactions on Image Processing, vol. 14, no. 10, pp. 1479–1485, 2005.
  • [7] Xiangyu Deng, Yide Ma, and Min Dong, “A new adaptive filtering method for removing salt and pepper noise based on multilayered pcnn,” Pattern Recognition Letters, vol. 79, pp. 8–17, 2016.
  • [8] Kenny Kal Vin Toh and Nor Ashidi Mat Isa, “Noise adaptive fuzzy switching median filter for salt-and-pepper noise reduction,” IEEE Signal Processing Letters, vol. 17, no. 3, pp. 281–284, 2010.
  • [9] Peixuan Zhang and Li Fang, “A new adaptive weighted mean filter for removing salt-and-pepper noise,” IEEE Signal Processing Letters, vol. 21, no. 10, pp. 1280–1283, 2014.
  • [10] Wang Yi, Jiangyun Wang, Song Xiao, and Han Liang, “An efficient adaptive fuzzy switching weighted mean filter for salt-and-pepper noise removal,” IEEE Signal Processing Letters, vol. 23, no. 11, pp. 1582–1586, 2016.
  • [11] UğUR ERKAN and LEVENT GöKREM, “A new method based on pixel density in salt and pepper noise removal,” Turkish Journal of Electrical Engineering and Computer Science, vol. 26, pp. 162–171, 2018.
  • [12] Uğur Erkan, Levent Gökrem, and Serdar Enginoğlu, “Different applied median filter in salt and pepper noise,” Computers & Electrical Engineering, vol. 70, pp. 789–798, 2018.
  • [13] A. Buades, B. Coll, and J. . Morel, “A non-local algorithm for image denoising,” in

    2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05)

    , June 2005, vol. 2, pp. 60–65 vol. 2.
  • [14] Jacques Froment, “Parameter-free fast pixelwise non-local means denoising,” Image Processing On Line, vol. 4, pp. 300–326, 2014.
  • [15] A. Hore and D. Ziou, “Image quality metrics: PSNR vs. SSIM,” in 2010 20th International Conference on Pattern Recognition, Aug 2010, pp. 2366–2369.
  • [16] Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, 2004.