Superpixel is widely used in image processing. And among the methods for superpixel generation, clustering-based methods have a high speed and a good performance at the same time. However, most clustering-based superpixel methods are sensitive to noise. To solve these problems, in this paper, we first analyze the features of noise. Then according to the statistical features of noise, we propose a novel centroid updating approach to enhance the robustness of the clustering-based superpixel methods. Besides, we propose a novel superpixel based edge detection method. The experiments on BSD500 dataset show that our approach can significantly enhance the performance of clustering-based superpixel methods in noisy environment. Moreover, we also show that our proposed edge detection method outperforms other classical methods.READ FULL TEXT VIEW PDF
This paper introduces an efficient edge detection method based on Gabor
Image edge information is very important in application areas such as ma...
Recently, the speaker clustering model based on aggregation hierarchy cl...
Traditional nearest points methods use all the samples in an image set t...
Occlusion edge detection requires both accurate locations and context
Acoustic beamformers have been widely used to enhance audio signals. The...
Polarimetric synthetic aperture radar (PolSAR) has achieved a prominent
In 2003, Ren et al.  first proposed the concept of superpixel, which refers to a series of regions consists of pixels with adjacent positions and similar color, brightness and texture features. These regions can retain the effective information like the boundary information of objects in the image [2, 3, 4]. Different from pixel, superpixel can reduce the size of the object to be processed and the complexity of the subsequent processing to a great extent [2, 3, 4]. Due to these advantages, superpixel methods are usually used as a preprocessing step for many tasks [2, 3, 4, 5].
For the past years, many superpixel methods have been proposed. A good superpixel method should meet many needs like compactness, boundary adherence, computational complexity, controllable superpixel number [2, 6]. Each kind of superpixel method has its own advantages and defects [2, 7]. Among them, clustering-based methods are widely used for image segmentation tasks [2, 8]. Through the clustering process, the number and the compactness of superpixels can be controlled 
. Simple linear iterative clustering (SLIC) is one of the most commonly used clustering-based methods, which adopts a local K-means clustering method to cluster pixels based on the color and spatial distance
. Linear spectral clustering (LSC) is another well-known clustering-based method. Different to the five-dimensional space used in SLIC, it takes a ten-dimensional space and gets a better boundary recall rate than SLIC. Recently, an improved SLIC called simple non-iterative clustering (SNIC) has been developed . Compared to SLIC and LSC, SNIC do not need iterations, so it has higher computational and memory efficiency. However, above clustering-based methods are all sensitive to noise . When the noise exists, they can not maintain the performance as they work in non-noise situation .
To solve the above problems of clustering-based superpixel methods, in this paper, we first analyze the features of noise. Then according to the statistical features of noise, we propose a novel centroid updating approach to enhance the clustering-based superpixel methods. Moreover, we propose a superpixel based edge detection algorithm (SBED), which can gain edge of image by detecting edges of superpixels. The contribution of this paper can be concluded as follows,
We analyze the reason why clustering-based superpixel methods don’t work well in noisy environment.
According to the features of noise, we propose a novel centroid update approach for clustering-based superpixel methods to reduce the impact of noise.
Based on superpixel segmentation, we propose a new edge detection method.
When noise exists, value of pixel tends to be singular in the image. While the cluster centroid usually takes the mean value of all pixels with corresponding label, the impact of noise will be accumulated in the cluster centroid. Hence noise will affect the cluster centroid, and as we know, the cluster centroid plays an important role in the process of clustering.
Take SLIC for example, as shown in the Fig. 1, in the early stage of iterative clustering in noise-free environment, the segmentation result has some false boundaries. Generally, the error will reduce and converge with the increase of iteration times. But in noisy environment, the existence of noise will lead to a worse false segmentation, and the false segmentation will then lead to more errors on the clustering centroids as a positive feedback. As Fig. 1 shows, in noisy environment, the difference of cluster centroids becomes ambiguity, and cluster centroids can’t capture the features of the expected object but the mixed region caused by error segmentation. Finally, it will results in a poor segmentation.
Distribution of most noise follows or approximates to the Gaussian distribution as follows:
where is the value of the pixel in an image, denotes the average or expected value of , and
denotes the standard deviation of.
As Eq. (1) shows, the value of is distributed on both sides of . Hence, most denoising methods take the neighbors of the pixel to eliminate the impact of noise, such as the works in [11, 12], they deal with the noisy pixel by taking the mean value of pixels within a square selected window centered at the current pixel.
Considering the statistical features of noise and inspired by these denoising methods, we propose a novel centroid update approach and apply it to the clustering-based superpixel methods. For an image with size and desired number of superpixels , can be computed as follows:
where represents the spatial part (spatial centroid) of the mixed centroid , is the color centroid, and represents the region of the th superpixel, is the number of pixels within , is the current coordinate value of pixel . Because the spatial centroid is not affected by noise, here we still use all the pixels within the current superpixel to calculate it. Different to the spatial space, the color space will be strongly affected by noise. So when calculate the color centroid of the th superpixel, instead of using all pixels in a superpixel, we use a square block centered at with an adaptive size to select the pixels and calculate it, here owns approximately half of the pixels within the th superpixel. There are two advantages to do like this: (1) it can reduce the effect of noise on the clustering centroids by taking the mean value of pixels within the square block; (2) it also avoids the error caused by false segmentation, most pixels within used to compute the color centroid can capture the features of the expected object instead of the mixed region caused by error segmentation.
An overview of our approach is shown in Fig. 2. And the approach can be used for clustering-based superpixel methods. Here for convenience we call our approach Centroid-X, indicates a specific clustering-based superpixel method, for example, Centroid-SLIC means that the SLIC method is enhanced by our Centroid-X approach.
Superpixels can well preserve the boundary of the object, so we can detect the edge of the image based on the edges of superpixels. And here, we use the relationship between superpixels to determine which edges should be reserved and which should be removed.
For a neighboring superpixel pair: superpixel and superpixel , the distance between them is defined as follow:
where , and is the value of pixel in CIELAB space.
Then we compute the adjacent matrix A of superpixels (A is the upper triangular matrix). The th row of A consists of the distance between the th superpixel and its neighbors. And for A, its mean value is the mean of its non-zero elements. If , the edge between and should be removed, otherwise it should be reserved.
Finally, we use the gradient of superpixel to further detect the edge point. The whole procedure of SBED is presented in Algorithm 1.
In this section, we apply our Centroid-X on three common used clustering-based superpixel methods: SLIC, LSC, and SNIC. We compare the performance of the original method and the enhanced method using our approach on the Berkeley benchmark (BSD500)  with three kinds of environment: noise-free, Gaussian noise (zero mean with standard deviation (std) range [0.1,0.2]), and salt and pepper noise (noise density range [0.1, 0.2]). All the work is run on a personal computer with Windows 10, Intel(R) Core(TM) i7 2.2 GHz 6 cores CPU, and 8 GB RAM 111Data and codes for our proposed centroid update approach and SBED are published on https://github.com/ProfHubert/ICASSP..
We select one standard metric for compactness and two standard metrics for boundary adherence: compactness metric (CO) , boundary recall rate (BR) , and under segmentation error (UE) . Higher BR and lower UE mean a more accurate segmentation, and higher CO means a better compactness of superpixels.
In the experiments, the parameters settings between and Centroid-X are the same. To keep the fairness, the compactness coefficient are all setting to the same level (SLIC and SNIC take compactness coefficient 30, LSC uses ratio 0.3).
Fig. 3 shows the BR, UE, and CO curves of all methods in noise-free environment. By comparing them, we can find that (SLIC, LSC, SNIC) obtains better BR than Centroid-X, but the enhanced one obtains better CO, and the UE between them is basically the same. Fig. 4 shows their running time, except SNIC, the speed between SLIC and Centroid-SLIC and the speed between LSC and Centroid-LSC have little difference. Because SNIC takes a non-iterative clustering and its main computation focuses on the centroid update, hence Centroid-X affect the computation of SNIC more than SLIC and LSC. Generally speaking, the difference between and Centroid-X is not significant in noise-free environment.
Fig. 5 shows the comparison of and Centroid-X in Gaussian noise environment. We can see that Centroid-SLIC and Centroid-SNIC obtain much better BR, UE and CO than SLIC and SNIC. Although LSC obtains better BR than Centroid-LSC, but its UE and CO are significantly worse than Centroid-LSC. In Fig .6, we can see that the performance of LSC in Gaussian noise environment are quite bad (like over-segmentation), so do SLIC and SNIC, while their corresponding Centroid-X can still maintain the approximate performance like in noise-free environment.
Fig. 7 shows the performance of and Centroid-X in Salt and Pepper noise environment. By observing Fig. 6, we can find that the number of the output superpixels of SLIC and LSC falls sharply (like under-segmentation) in Salt and Pepper noise environment than noise-free environment. But Centroid-SLIC and Centroid-LSC can still maintain comparable number of output superpixels. Although CO of Centroid-X is slightly weaker than , Centroid-X still obtains much better BR and UE, which illustrates that even in Salt and Pepper noise environment, Centroid-X can still get a robust performance like in noise-free environment.
as evaluation metrics like on BSD500. Here we set , and . For Sobel and Canny, we set threshold as 0.1.
In this paper, we propose a novel centroid updating approach to enhance the clustering-based superpixel methods and a superpixel based edge detection method. Experiments illustrate that our proposed methods can get a much better performance in noisy environment compared state-of-the-art methods.
Proceedings of the IEEE International Conference on Computer Vision. IEEE, 2003, p. 10.
“Superpixel tensor pooling for visual tracking using multiple midlevel visual cues fusion,”IEEE Access, pp. 1–1, 2019.
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017, pp. 4651–4660.