A novel centroid update approach for clustering-based superpixel method and superpixel-based edge detection

10/18/2019 ∙ by Houwang Zhang, et al. ∙ 0

Superpixel is widely used in image processing. And among the methods for superpixel generation, clustering-based methods have a high speed and a good performance at the same time. However, most clustering-based superpixel methods are sensitive to noise. To solve these problems, in this paper, we first analyze the features of noise. Then according to the statistical features of noise, we propose a novel centroid updating approach to enhance the robustness of the clustering-based superpixel methods. Besides, we propose a novel superpixel based edge detection method. The experiments on BSD500 dataset show that our approach can significantly enhance the performance of clustering-based superpixel methods in noisy environment. Moreover, we also show that our proposed edge detection method outperforms other classical methods.



There are no comments yet.


page 2

page 4

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

In 2003, Ren et al. [1] first proposed the concept of superpixel, which refers to a series of regions consists of pixels with adjacent positions and similar color, brightness and texture features. These regions can retain the effective information like the boundary information of objects in the image [2, 3, 4]. Different from pixel, superpixel can reduce the size of the object to be processed and the complexity of the subsequent processing to a great extent [2, 3, 4]. Due to these advantages, superpixel methods are usually used as a preprocessing step for many tasks [2, 3, 4, 5].

For the past years, many superpixel methods have been proposed. A good superpixel method should meet many needs like compactness, boundary adherence, computational complexity, controllable superpixel number [2, 6]. Each kind of superpixel method has its own advantages and defects [2, 7]. Among them, clustering-based methods are widely used for image segmentation tasks [2, 8]. Through the clustering process, the number and the compactness of superpixels can be controlled [9]

. Simple linear iterative clustering (SLIC) is one of the most commonly used clustering-based methods, which adopts a local K-means clustering method to cluster pixels based on the color and spatial distance


. Linear spectral clustering (LSC) is another well-known clustering-based method

[10]. Different to the five-dimensional space used in SLIC, it takes a ten-dimensional space and gets a better boundary recall rate than SLIC. Recently, an improved SLIC called simple non-iterative clustering (SNIC) has been developed [5]. Compared to SLIC and LSC, SNIC do not need iterations, so it has higher computational and memory efficiency. However, above clustering-based methods are all sensitive to noise [2]. When the noise exists, they can not maintain the performance as they work in non-noise situation [2].

To solve the above problems of clustering-based superpixel methods, in this paper, we first analyze the features of noise. Then according to the statistical features of noise, we propose a novel centroid updating approach to enhance the clustering-based superpixel methods. Moreover, we propose a superpixel based edge detection algorithm (SBED), which can gain edge of image by detecting edges of superpixels. The contribution of this paper can be concluded as follows,

  • We analyze the reason why clustering-based superpixel methods don’t work well in noisy environment.

  • According to the features of noise, we propose a novel centroid update approach for clustering-based superpixel methods to reduce the impact of noise.

  • Based on superpixel segmentation, we propose a new edge detection method.

2 Methods

2.1 A novel centroid update approach

When noise exists, value of pixel tends to be singular in the image. While the cluster centroid usually takes the mean value of all pixels with corresponding label, the impact of noise will be accumulated in the cluster centroid. Hence noise will affect the cluster centroid, and as we know, the cluster centroid plays an important role in the process of clustering.

Figure 1: Performance of SLIC in non-noise environment and Gaussian noise environment with different iteration times.

Take SLIC for example, as shown in the Fig. 1, in the early stage of iterative clustering in noise-free environment, the segmentation result has some false boundaries. Generally, the error will reduce and converge with the increase of iteration times. But in noisy environment, the existence of noise will lead to a worse false segmentation, and the false segmentation will then lead to more errors on the clustering centroids as a positive feedback. As Fig. 1 shows, in noisy environment, the difference of cluster centroids becomes ambiguity, and cluster centroids can’t capture the features of the expected object but the mixed region caused by error segmentation. Finally, it will results in a poor segmentation.

Distribution of most noise follows or approximates to the Gaussian distribution as follows:


where is the value of the pixel in an image, denotes the average or expected value of , and

denotes the standard deviation of


As Eq. (1) shows, the value of is distributed on both sides of . Hence, most denoising methods take the neighbors of the pixel to eliminate the impact of noise, such as the works in [11, 12], they deal with the noisy pixel by taking the mean value of pixels within a square selected window centered at the current pixel.

Considering the statistical features of noise and inspired by these denoising methods, we propose a novel centroid update approach and apply it to the clustering-based superpixel methods. For an image with size and desired number of superpixels , can be computed as follows:


where represents the spatial part (spatial centroid) of the mixed centroid , is the color centroid, and represents the region of the th superpixel, is the number of pixels within , is the current coordinate value of pixel . Because the spatial centroid is not affected by noise, here we still use all the pixels within the current superpixel to calculate it. Different to the spatial space, the color space will be strongly affected by noise. So when calculate the color centroid of the th superpixel, instead of using all pixels in a superpixel, we use a square block centered at with an adaptive size to select the pixels and calculate it, here owns approximately half of the pixels within the th superpixel. There are two advantages to do like this: (1) it can reduce the effect of noise on the clustering centroids by taking the mean value of pixels within the square block; (2) it also avoids the error caused by false segmentation, most pixels within used to compute the color centroid can capture the features of the expected object instead of the mixed region caused by error segmentation.

Figure 2: An overview of our proposed approach.

An overview of our approach is shown in Fig. 2. And the approach can be used for clustering-based superpixel methods. Here for convenience we call our approach Centroid-X, indicates a specific clustering-based superpixel method, for example, Centroid-SLIC means that the SLIC method is enhanced by our Centroid-X approach.

2.2 Our proposed superpixel based edge detection

Superpixels can well preserve the boundary of the object, so we can detect the edge of the image based on the edges of superpixels. And here, we use the relationship between superpixels to determine which edges should be reserved and which should be removed.

For a neighboring superpixel pair: superpixel and superpixel , the distance between them is defined as follow:


where , and is the value of pixel in CIELAB space.

Then we compute the adjacent matrix A of superpixels (A is the upper triangular matrix). The th row of A consists of the distance between the th superpixel and its neighbors. And for A, its mean value is the mean of its non-zero elements. If , the edge between and should be removed, otherwise it should be reserved.

Finally, we use the gradient of superpixel to further detect the edge point. The whole procedure of SBED is presented in Algorithm 1.

0:  Image I, number of superpixels
0:  Edge matrix E
  Segment I into superpixels and then detect the edges of superpixels;
  Compute gradient matrix G as Sobel [17], set ;
  Compute A using Eq. 3 and gain the average of A;
  for to do
    for to do
      if then       Eliminate the edge of E between and ;
      end if
    end for
  end for
  for each element do
    if then ; end if
    if then ; end if
   end for
  return E;
Algorithm 1 Superpixel based edge detection

3 Experiments

In this section, we apply our Centroid-X on three common used clustering-based superpixel methods: SLIC, LSC, and SNIC. We compare the performance of the original method and the enhanced method using our approach on the Berkeley benchmark (BSD500) [13] with three kinds of environment: noise-free, Gaussian noise (zero mean with standard deviation (std) range [0.1,0.2]), and salt and pepper noise (noise density range [0.1, 0.2]). All the work is run on a personal computer with Windows 10, Intel(R) Core(TM) i7 2.2 GHz 6 cores CPU, and 8 GB RAM 111Data and codes for our proposed centroid update approach and SBED are published on https://github.com/ProfHubert/ICASSP..

3.1 Evaluation metrics and parameter settings

We select one standard metric for compactness and two standard metrics for boundary adherence: compactness metric (CO) [14], boundary recall rate (BR) [15], and under segmentation error (UE) [16]. Higher BR and lower UE mean a more accurate segmentation, and higher CO means a better compactness of superpixels.

In the experiments, the parameters settings between and Centroid-X are the same. To keep the fairness, the compactness coefficient are all setting to the same level (SLIC and SNIC take compactness coefficient 30, LSC uses ratio 0.3).

3.2 Results and analysis

Figure 3: The comparison of and Centroid-X in noise-free environment.

Figure 4: The running time of and Centroid-X in noise-free environment.

Fig. 3 shows the BR, UE, and CO curves of all methods in noise-free environment. By comparing them, we can find that (SLIC, LSC, SNIC) obtains better BR than Centroid-X, but the enhanced one obtains better CO, and the UE between them is basically the same. Fig. 4 shows their running time, except SNIC, the speed between SLIC and Centroid-SLIC and the speed between LSC and Centroid-LSC have little difference. Because SNIC takes a non-iterative clustering and its main computation focuses on the centroid update, hence Centroid-X affect the computation of SNIC more than SLIC and LSC. Generally speaking, the difference between and Centroid-X is not significant in noise-free environment.

Figure 5: The comparison of and Centroid-X in Gaussian noise environment (level means the std of the noise).

Figure 6: The visual comparison of and Centroid-X in noise-free environment and noisy environment (the desired number of superpixels for all methods is set as 600).

Fig. 5 shows the comparison of and Centroid-X in Gaussian noise environment. We can see that Centroid-SLIC and Centroid-SNIC obtain much better BR, UE and CO than SLIC and SNIC. Although LSC obtains better BR than Centroid-LSC, but its UE and CO are significantly worse than Centroid-LSC. In Fig .6, we can see that the performance of LSC in Gaussian noise environment are quite bad (like over-segmentation), so do SLIC and SNIC, while their corresponding Centroid-X can still maintain the approximate performance like in noise-free environment.

Figure 7: The comparison of and Centroid-X in Salt and Pepper noise environment (level means the density of the noise).

Fig. 7 shows the performance of and Centroid-X in Salt and Pepper noise environment. By observing Fig. 6, we can find that the number of the output superpixels of SLIC and LSC falls sharply (like under-segmentation) in Salt and Pepper noise environment than noise-free environment. But Centroid-SLIC and Centroid-LSC can still maintain comparable number of output superpixels. Although CO of Centroid-X is slightly weaker than , Centroid-X still obtains much better BR and UE, which illustrates that even in Salt and Pepper noise environment, Centroid-X can still get a robust performance like in noise-free environment.

4 Application

Here, we apply Centroid-SLIC into our SBED to generate superpixel, and we compare SBED with classical edge detection methods like Sobel [17] and Canny [18]. We take PSNR [19] and SSIM [20]

as evaluation metrics like

[21] on BSD500. Here we set , and . For Sobel and Canny, we set threshold as 0.1.

Our method Sobel Canny
PSNR 6.8372 6.5926 6.1129
SSIM 0.0117 0.0112 0.0104
Table 1: Comparison of three edge detection methods.

Figure 8: The visual comparison of three edge detection methods in noise-free and noisy environment (in SBED, is set as 1500).

Table 1 shows that our method gets better PSNR and SSIM than Sobel and Canny. And Fig. 8 shows that our method can better obtain the edge of image in both noise-free and noisy environment.

5 Conclusion

In this paper, we propose a novel centroid updating approach to enhance the clustering-based superpixel methods and a superpixel based edge detection method. Experiments illustrate that our proposed methods can get a much better performance in noisy environment compared state-of-the-art methods.


  • [1] Xiaofeng Ren and Jitendra Malik, “Learning a classification model for segmentation,” in

    Proceedings of the IEEE International Conference on Computer Vision

    . IEEE, 2003, p. 10.
  • [2] C. Wu, L. Zhang, H. Zhang, and H. Yan, “Superpixels using fuzzy simple linear iterative clustering and fast precise number control,” arXiv, p. 1812.10932v2, 2018.
  • [3] C. Wu, L. Zhang, H. Zhang, and H. Yan, “Improved superpixel-based fast fuzzy c-means clustering for image segmentation,” in 2019 IEEE International Conference on Image Processing (ICIP), Sep. 2019, pp. 1455–1459.
  • [4] C. Wu, L. Zhang, J. Cao, and H. Yan,

    “Superpixel tensor pooling for visual tracking using multiple midlevel visual cues fusion,”

    IEEE Access, pp. 1–1, 2019.
  • [5] Radhakrishna Achanta and Sabine Susstrunk, “Superpixels and polygons using simple non-iterative clustering,” in

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition

    , 2017, pp. 4651–4660.
  • [6] Radhakrishna Achanta, Appu Shaji, Kevin Smith, Aurelien Lucchi, Pascal Fua, and Sabine Süsstrunk, “Slic superpixels compared to state-of-the-art superpixel methods,” IEEE transactions on pattern analysis and machine intelligence, vol. 34, no. 11, pp. 2274–2282, 2012.
  • [7] Varun Jampani, Deqing Sun, Ming-Yu Liu, Ming-Hsuan Yang, and Jan Kautz, “Superpixel sampling networks,” in Proceedings of the European Conference on Computer Vision (ECCV), 2018, pp. 352–368.
  • [8] Yuwei Guo, Licheng Jiao, Shuang Wang, Shuo Wang, Fang Liu, and Wenqiang Hua, “Fuzzy superpixels for polarimetric sar images classification,” IEEE Transactions on Fuzzy Systems, vol. 26, no. 5, pp. 2846–2860, 2018.
  • [9] David Stutz, Alexander Hermans, and Bastian Leibe, “Superpixels: An evaluation of the state-of-the-art,” Computer Vision and Image Understanding, vol. 166, pp. 1–27, 2018.
  • [10] Zhengqin Li and Jiansheng Chen, “Superpixel segmentation using linear spectral clustering,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2015, pp. 1356–1363.
  • [11] Antoni Buades, Bartomeu Coll, and J-M Morel, “A non-local algorithm for image denoising,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05). IEEE, 2005, vol. 2, pp. 60–65.
  • [12] Peixuan Zhang and Fang Li, “A new adaptive weighted mean filter for removing salt-and-pepper noise,” IEEE signal processing letters, vol. 21, no. 10, pp. 1280–1283, 2014.
  • [13] Pablo Arbelaez, Michael Maire, Charless Fowlkes, and Jitendra Malik, “Contour detection and hierarchical image segmentation,” IEEE transactions on pattern analysis and machine intelligence, vol. 33, no. 5, pp. 898–916, 2010.
  • [14] Alexander Schick, Mika Fischer, and Rainer Stiefelhagen, “An evaluation of the compactness of superpixels,” Pattern Recognition Letters, vol. 43, pp. 71–80, 2014.
  • [15] Alex Levinshtein, Adrian Stere, Kiriakos N Kutulakos, David J Fleet, Sven J Dickinson, and Kaleem Siddiqi, “Turbopixels: Fast superpixels using geometric flows,” IEEE transactions on pattern analysis and machine intelligence, vol. 31, no. 12, pp. 2290–2297, 2009.
  • [16] Olga Veksler, Yuri Boykov, and Paria Mehrani, “Superpixels and supervoxels in an energy optimization framework,” in European conference on Computer vision. Springer, 2010, pp. 211–224.
  • [17] Nick Kanopoulos, Nagesh Vasanthavada, and Robert L Baker, “Design of an image edge detection filter using the sobel operator,” IEEE Journal of solid-state circuits, vol. 23, no. 2, pp. 358–367, 1988.
  • [18] John Canny, “A computational approach to edge detection,” IEEE Transactions on pattern analysis and machine intelligence, , no. 6, pp. 679–698, 1986.
  • [19] Alain Hore and Djemel Ziou, “Image quality metrics: Psnr vs. ssim,” in 2010 20th International Conference on Pattern Recognition. IEEE, 2010, pp. 2366–2369.
  • [20] Zhou Wang, Alan C Bovik, Hamid R Sheikh, Eero P Simoncelli, et al., “Image quality assessment: from error visibility to structural similarity,” IEEE transactions on image processing, vol. 13, no. 4, pp. 600–612, 2004.
  • [21] Y He and LM Ni, “A novel scheme based on the diffusion to edge detection.,” IEEE transactions on image processing: a publication of the IEEE Signal Processing Society, vol. 28, no. 4, pp. 1613–1624, 2019.