I Introduction
Morphological reconstruction (MR) [1] is a powerful operation in mathematical morphology. It has been widely used in image filtering [2], image segmentation [3], and feature extraction [4], etc. Among these applications, one of the most important applications is that MR is often used in seeded segmentation algorithms [5], [6] such as watershed transformation (WT) [7] and power watershed (PW) [8] to reduce oversegmentation caused by image noise and details. However, there are two drawbacks [9], [10] when MR is used in seeded segmentation algorithms.

It is difficult to reduce oversegmentation while obtaining a high segmentation accuracy for seeded segmentation algorithms (we use MRWT to denote MRbased watershed transform and use MRPW to denote MRbased power watershed). Although MR is able to filter noise in gradient images, some important contour details are smoothed out as well.

MR is sensitive to the scale of structuring elements. In practical applications, if the scale is too small, the reconstructed gradient image suffers from a serious oversegmentation. Oppositely, if the scale is too large, the reconstructed gradient image suffers from an undersegmentation.
Generally, MR is used in watershed transform to improve the segmentation effect by employing a structuring element to filter regional minima [11]. However, it is very difficult to filter useless regional minima while preserving meaningful ones by simply considering one singlescale structuring element. Although
min imposition [12] is a simple and efficient method for oversegmentation reduction, it relies on a threshold choice and is likely to miss some important boundaries. Region merging [13], [14] is also a popular method for this, but it requires iterating and renewing edge weight leading to a high computing burden. In addition, some researchers employ reasonable contour detection methods, e.g., globalized probability of boundary (gPb) [15] that combines the multiscale information from brightness, color and texture, to achieve better image segmentation. However, the gPb is computationally expensive because it combines too many feature cues for contour detection. To speed up the algorithm of contour detection, Dollar and Zitnick [16] took the advantage of the structure present in regional image patches and random decision forests, and proposed a fast structured edge (SE) detection approach using structured forests. This algorithm obtains realtime performance and stateoftheart edge detection but requires a huge amount of memory for training data. To reduce memory requirement, Hallman and Fowlkes [17] proposed a simple and efficient model to learn contour detection, namely oriented edge forests (OEF). Although these improved contour detectors are superior to traditional detectors, e.g. Sobel or Canny, and they are helpful for improving subsequent image segmentation, they still generate a large number of seeds leading to serious oversegmentations.
In practice, contour detection methods are usually combined with other approaches to improve image segmentation effect. For example, Fu et al.
[18] proposed a robust image segmentation approach using contourguided color palettes by integrating contour and color cues, where SE, meanshift algorithm [19], region merging, and spectral clustering [20] are combined to achieve better segmentation results. However, the approach is complex because it combines several different algorithms that requires many parameters.
In this paper, we propose an adaptive morphological reconstruction (AMR) operation that is able to generate a better seed image than MR to improve seeded segmentation algorithms. Firstly, AMR employs multiscale structuring elements to reconstruct a gradient image. Secondly, a pointwise maximum operation on these reconstructed gradient images is performed to obtain the final adaptive reconstruction result. Because AMR employs small structuring elements to reconstruct pixels of large gradient magnitudes while employing large structuring elements to reconstruct pixels of small gradient magnitudes in a gradient image, AMR is able to obtain better seed images to improve the seeded segmentation algorithms. Our main contributions are summarized as follows.

Multiscale structuring elements are employed by AMR, and different scaled structuring elements are adaptively adopted by pixels of different gradient magnitudes without computing the local features of a gradient image.

AMR has a convergence property and a monotonic increasing property, the two properties help seeded segmentation algorithms to achieve a hierarchical segmentation.

AMR has a low computational complexity, and it can help seedbased spectral segmentation to achieve better image segmentation results than thestateofart algorithms.
The rest of the paper is organized as follows. In the next section, the research background related with AMR is introduced and analyzed. On this basis, AMR is proposed, and its two properties, monotonic increasingness and convergence are carefully analyzed in Section III. To demonstrate the superiority of AMR, AMR is used for seeded image segmentation and seedbased spectral segmentation. Experiments are presented in Section IV, followed by the conclusion in Section V.
Ii Background
Iia Morphological Reconstruction
MR is an image transformation that requires two input images, a marker image and a mask image. Let two grayscale images and denote the marker image that is the starting point for the transformation and the mask image that constrains the transformation, respectively [21]. If , which means is pointwise less than or equal to , the morphological dilation reconstruction of from is denoted by
(1) 
where , for , satisfies . The symbol represents the elementary morphological dilation operation, and stands for the pointwise minimum at each pixel of two images as shown in Fig. 1(a).
Similarly, if , the morphological erosion reconstruction of from , which is the dual operation of , is defined as
(2) 
where , for , satisfies . The symbol represents the elementary morphological erosion operation, and stands for the pointwise maximum at per pixel of two images as shown in Fig. 1(b).
To further illustrate the principle of MR for image transformation, we present an example for the binary MR as shown in Fig. 2, where the red regions denote seeds, i.e., the marker image .
According to Fig. 2 and (1)(2), a suitable marker image is important for MR. We have known that for while for . Thus, there are lots of choices for . Different marker images corresponds to different reconstruction results as shown in Fig. 3. To obtain an efficient in practice, the marker image is usually obtained by performing a transformation on the corresponding mask image [22][24]. For example, the erosion or dilation result of a mask image is often considered as a marker image [25], i.e., or , where is a disk shaped structuring element, the radius of is , . Therefore, MR is sensitive to the parameter because the marker image is decided by the scale of the structuring element.
As compositional morphological opening and closing operations show better performance than elementary morphological erosion and dilation operations for image filtering, feature extraction, etc., we present the definition of compositional morphological opening and closing reconstructions ( and ) of from as follows
(3) 
IiB Multiscale and Adaptive Mathematical Morphology
For image filtering and enhancement using morphological operators, a largescale structuring element can suppress noise but may also blur the image details, whereas a smallscale structuring element can preserve image details but may fail to suppress noise. Some researchers proposed multiscale and adaptive morphological operators to improve the performance of traditional morphological operators. However, most multiscale morphological operators [26], [27] such as morphological gradient operators and morphological filtering operators, average all scales of morphological operation results as final output
(4) 
where is the final output result, is the radius of the structuring element, , , . Although the average result is superior to the result based on singlescale morphological operators, it causes contour offset and mistakes. Some researchers improved multiscale morphological operators by introducing a weighted coefficient to (4), and they defined adaptive multiscale morphological operators as follows [28]
(5) 
where is the weighted coefficient on the th scale result. However, because the computing of weighted coefficients is complex, the adaptive multiscale morphological operators have a low computational efficiency. Moreover, the weighted average result is similar to average result because it is difficult to obtain the optimal weighted coefficient, even though the former is slightly better than the latter.
Although lots of adaptive multiscale morphological operators [29][32] have been proposed, it can be seen from (4)(5) that both the multiscale and adaptive morphological operators employ a linear combination of differentscale results to improve singlescale morphological gradient or filtering operators. Because the linear combination is unsuitable for multiscale morphological reconstruction operation, in this paper, we try to employ a nonlinear combination (i.e., the pointwise maximum operation denoted by ) to design adaptive morphological reconstruction operators. These operators are different from conventional multiscale and adaptive morphological operators employing linear combination in (4)(5). We use nonlinear operation instead of linear combination since the former is more suitable than the later for the removal of useless seeds in seeded image segmentation.
IiC Seeded Segmentation
Seeded segmentation algorithms, such as graph cuts [33], random walker [34], watersheds [7], and power watershed [8] have been widely used in complex image segmentation tasks due to their good performance [35]. It is not required to give seed images for both graph cuts and random walker because they usually consider each pixel as a seed. However, a seed image is necessary for WT and PW by computing the regional minima of a gradient image.
Since both WT and PW obtain seeds from a gradient image that often includes a huge number of seeds generated by noise and unimportant texture details, they usually suffer from oversegmentation. A larger number of approaches for addressing oversegmentation was proposed, and these approaches can be categorized into two groups.

Feature extraction or feature learning is used to obtain a better gradient image that enhances important contours while smoothing noise and texture details [15][17].

MR is used for gradient reconstruction to reduce the number of regional minima [36][38].
For the first group of approaches, gPb, OEF, and SE are popular for reducing oversegmentation as shown in Figs. 45. In Fig. 5, although gPb, OEF, and SE provide better gradient images that can reduce oversegmentation for WT and PW, the segmentation results are still poor compared to ground truths shown in Fig. 4.
The second group of approaches depends on MR and WT, it is denoted by MRWT. Najman and Schmitt [36] employed MR to remove regional minima to reduce oversegmentation. Furthermore, a dynamic threshold is used to change the gradient magnitude that is smaller than the threshold, and then a hierarchical segmentation result is obtained. Wang [26] proposed a multiscale morphological gradient algorithm (MMG) for image segmentation using watersheds. The proposed MMG employs multiple structuring elements to obtain a better gradient image, and uses MR to remove regional minima to improve watershed segmentation.
Fig. 6 illustrates the principle of a seeded segmentation framework based on MRWT. We can see that the number of the regional minima in the gradient image decreases rapidly with the increase of , but the boundary is also destroyed simultaneously. It is clear that the larger structuring element corresponds to fewer seeds. One major reason is that MR employs a singlescale structuring element, which equally treats all pixels of different gradient magnitudes in the gradient image. For example, in dilation reconstruction, the marker image converges to the minimum grayscale value of pixels in the mask image as the value of increases. Obviously, both large and small structuring elements lead to poor reconstruction results while a moderatesized structuring element achieves a rough balance via sacrificing contour precision. Therefore, it is difficult to obtain a good seed image by employing a singlescale structuring element. Although many researchers employ multiscale structuring elements to generate a better gradient image, there are few studies on multiscale MR for gradient images. Moreover, the fusion of differentscale results is also a problem.
IiD Spectral Segmentation
It is wellknown that spectral clustering [20] is greatly successful due to the fact that it does not make strong assumptions on data distribution, and it is implemented efficiently even for large datasets, as long as we make sure that the affinity matrix is sparse. However, since the size of the affinity matrix is
for an image of size, and it is not sparse because of Gaussian similarity measure, spectral clustering is often inefficient for image segmentation due to eigenvalue decomposition of the huge affinity matrix. To address the issue, a great number of algorithms have been proposed to construct a smaller affinity matrix and thus to improve the computational efficiency of spectral clustering [39][42]. Most of these algorithms employ presegmentation (superpixel) methods such as the simple linear iterative clustering (SLIC) [43], meanshift [19], linear spectral clustering (LSC) [44], and superpixel hierarchy [45], to reduce the number of pixels of the original image and, in turn, reduces the size of the affinity matrix. As an example, Zhang
et al. [46] proposed a fast image segmentation approach that is a reexamination of spectral clustering on image segmentation. The approach provides better image segmentation results yet requires a long running time.The popular superpixel approaches have some drawbacks for spectral segmentation. Firstly, meanshift algorithm involves three parameters and it is sensitive to these parameters. Secondly, SLIC only generates superpixels that include regular regions, and these regions have a similar shape and size. Finally, LSC is superior to SLIC because LSC successfully connects a local feature with a global optimization objective function, so that LSC can generate more reasonable segmentation results. However, similar to SLIC, LSC also provides superpixels that include regular regions with a similar shape and size.
As seedbased spectral segmentation algorithms are sensitive to presegmentation results, an excellent presegmentation algorithm can improve segmentation results generated by seedbased spectral segmentation algorithms.
Iii Adaptive morphological reconstruction
Iiia The Proposed AMR
To overcome the drawback of MR on regional minima filtering, we propose an AMR that is able to filter useless regional minima and maintains meaningful ones generated by salient objects. Fig. 7 shows the motivation of AMR in which multiscale structuring elements are employed to reconstruct a gradient image, i.e., small structuring elements are adopted by pixels of large gradient magnitude while large structuring elements are adopted by pixels of small gradient magnitude.
Definition 1. Let be a series of nested structuring elements, where is the scale parameter of a structuring element, , . For a gradient image such that and , the adaptive morphological reconstruction denoted by of from is defined as
(6) 
Note that the pointwise maximum operation is only suitable for , but not suitable for . Because (the proof is presented in Appendix A) and , is unable to obtain a significantly convergent gradient image if .
We apply AMR to the gradient image shown in Fig. 6. The reconstruction and segmentation results are shown in Fig. 8, where the adopted structuring elements are disk and . More detailed comparisons are shown in Fig. 9. By comparing Fig. 8 with Fig. 6, it is obvious that AMR obtains better seed images than MR due to the fact that the nonlinear operation is able to remove efficiently useless seeds.
To further show the influence of on AMR, Fig. 10 shows segmentation results provided by AMR through changing the value of . We can see that there are some small segmented areas when the value of is small. These small areas are merged by increasing the value of . However, although a large leads to the merge of small areas, the precision of object contours will be decreased as shown in Fig. 6. Therefore, we usually set for a moderatesized image.
IiiB The Monotonic Increasingness Property of AMR
AMR is an algorithm that aims at finding meaningful regional minima by merging or filtering useless regional minima. AMR includes two parameters and . When we increase the value of , gradient images reconstructed by AMR keep the increasing order as shown in Theorem 1.
Theorem 1. Let be an adaptive morphological reconstruction operator, is increasing with respect to the scale of structuring elements, i.e., for a gradient image such that and , , , , we have
(7) 
The proof of Theorem 1 is presented in Appendix B. Theorem 1 shows that the gradient image processed by AMR is monotonous increasing with the increase of . Fig. 9 demonstrates Theorem 1. We can see that if is enlarged, the more unimportant seeds are removed, and important seeds are preserved. Actually, the result is equivalent to region merging. However, the method is simpler than region merging. According to the result, it can be seen that AMR can help seeded segmentation algorithms to achieve a hierarchical segmentation [47], [48]. Hierarchical segmentation is a multilevel segmentation scheme, and it usually outputs a coarsetofine hierarchy of segments ordered by the level of details. Multiscale combinatorial grouping (MCG) proposed by PontTuset et al. [49] is an excellent hierarchical segmentation approach that employs a fast normalized cut algorithm and an efficient algorithm for combinatorial merging of hierarchical regions. Based on the hierarchical segmentation results provided by MCG, some improved approaches are also proposed [50], [51]. These improved approaches achieve better segmentation effect but have lower computational efficiency than MCG.
Before analyzing the relationship between AMRWT and hierarchical segmentation, we first review some basic concepts of hierarchical segmentation. Let be a finite set. A hierarchy on is a set of parts of such that

.

For every , .

For each pair , or .
Note that is a chain of nested partitions. Let be the initial partition of , which corresponds to the finest partition of , and be the coarsest partition of , which segments the images as one single region. A partition , , on has the property that
(8) 
(9) 
(10) 
where denotes the partition. is finer than the partition . Derived from Theorem 1 and Fig. 9, we obtain
(11) 
where denotes seeded segmentation algorithms such as WT or PW. Suppose that , , and then
(12) 
According to (12), the principle of the hierarchical segmentation based on AMR is shown in Fig. 11, in which the data points represent regions obtained by the hierarchical segmentation at different levels.
IiiC The Convergence Property of AMR
By comparing Fig. 6 with Fig. 8, it can be observed that AMR provides significant gradient images and AMRWT generates convergent segmentation results via enlarging the scale of structuring elements. An important convergence property of AMR is described in the following.
Theorem 2. Let be an adaptive morphological reconstruction operator, is convergent when increasing the scale parameter , i.e., for any gradient images and such that , if then
(13) 
i.e., , and the proof is presented in Appendix C.
According to Fig 9, it can be seen that the gradient result and the corresponding seed image will remain unchanged when . This empirically illustrates that the gradient image reconstructed by AMR is convergent when increasing the value of . Besides, the large gradient magnitude is unchanged while the small gradient magnitude converges to ones larger than itself for AMR. However, the large gradient magnitude converges to one smaller than itself while the small gradient magnitude converges to one larger than itself for MR when the structuring element is small. With the increase of the value of , the value of gradient magnitudes finally converges to the minimum of the original gradient image, i.e., (see Appendix A). Consequently, MR removes all regional minima while AMR only filters useless regional minima and preserves significant ones when .
Furthermore, we analyze how to determine the parameter for AMR. The computational efficiency of AMR is influenced by the parameter . A small means a low computational complexity. According to Theorem 2, the reconstructed gradient image and the corresponding segmentation result are unchanged when , but the obtained is usually large. As the paper aims at employing AMR to improve seeded segmentation algorithms, we replace the convergence condition with checking the difference between and . We propose an objective function for justifying the convergence of AMR
(14) 
where , . It is clear that the segmentation result will remain unchanged when , is a minimal threshold error, and it is a constant used for , but is a variant for . Consequently, only a parameter needs to be tuned for obtaining different reconstruction results.
IiiD The Algorithm of AMR
AMR only involves the parameter and , as described in the detailed steps of AMR in Algorithm 1^{1}^{1}1Source code is available at https://github.com/SUSTreynole/AMR. To speed up the convergence of Algorithm 1, the three parameters , , and are used for AMR because the iteration can be stopped according to or . The computational complexity of AMR depends on the values of or . A large value of corresponds to a small value of . The larger is the value of , the longer is the execution time of AMR. Since we have known that AMR has a fast convergent property as shown in Fig. 8, a small is enough for moderatesized images in practical applications. A small m indicates that AMR has a low computational complexity.
Note that the parameter is unnecessary theoretically, we use two convergent condition and to speed up the convergence of Algorithm 1. We applied Algorithm 1 to images with complex texture content to demonstrate that the proposed AMR is effective for reducing oversegmentation as shown in Fig. 12. AMRWT not only overcomes the problem of oversegmentation but also obtains better contours than MRWT and stateoftheart superpixel methods. Furthermore, we test Algorithm 1 on images with text to show the monotonic increasing and convergent properties of AMR. Fig. 13 shows the comparison results. We can see that the segmentation results are nested, which demonstrates the monotonic increasing property of AMR. Moreover, the segmentation results are unchanged when , which demonstrates the convergent property of AMR.
Iv Experiments
To demonstrate the effectiveness and efficiency of the proposed AMR, we apply AMR to seeded image segmentation and spectral segmentation. We conduct experiments on the BSDS500 dataset. The experiments are performed on a workstation with an Intel Core (TM) i76700, 3.4GHz CPU and 16GB memory.
We compare the proposed algorithms with stateoftheart algorithms including a multiscale morphological gradient for watersheds (MMGWT) [26], multiscale ncut (MNCut) [52], orientedwatershed transformultrametric contour map (gPbowtucm) [15], the algorithm recovering occlusion boundaries from an image proposed by Hoiem (gPbHoiem) [53], spectral segmentation algorithms proposed by Kim et al. (FNCut, cPbowtucm) [39], Higherorder correlation clustering (HOCC) [54], global/regional affinity graph (GLgraph) [55], singlescale combinatorial grouping (SCG) [49], and multiscale combinatorial grouping (MCG) [49]. The open source codes and model parameters suggested by the corresponding authors are used. Because the author did not present specific parameter values for MMGRWT, we set and , where is a threshold and it is used to generate a marker image, and is the radius of the structuring element used for MR. For the proposed approaches, we set , , and .
We report the experimental results using three evaluation metrics to quantitatively measure the performance of segmentation algorithms: probabilistic rand index (PRI), segmentation covering (CV), and variation of information (VI). The PRI and CV are similarity measures, and they are large while the VI is small when the final segmentation is close to ground truth segmentation.
Iva Seeded Image Segmentation
AMR is useful for improving seeded image segmentation because it employs multiscale structuring elements to obtain a convergent seed image without presetting many parameters. To show the capability of AMR, it is applied to different gradient images to filter seeds. Fig. 14 shows reconstructed gradient images by AMR and the corresponding segmentation results by WT/PW. These results are clearly better than the ones shown in Fig. 5. The problem of oversegmentation for seeded segmentation algorithms is therefore addressed. Furthermore, compared Fig. 6 to Fig. 14, although both MR and AMR are able to filter seeds, AMR is able to maintain meaningful seeds that correspond to important contours.
Furthermore, Table I shows the number of seeds generated by gradient images. We can see that the reconstructed gradient images generate fewer seeds than original gradient images, which demonstrates AMR is efficient for the filtering of useless seeds. Moreover, AMR is robust for different gradient images obtained by Sobel, gPb, OEF, and SE because the final segmentation results are similar.
In Fig. 14, we set because the segmentation result includes too many small regions when . Clearly, controls the number of small regions in segmentation results. Generally, the value of depends on the resolution of the images to be segmented, e.g., for BSDS500.
To demonstrate that the proposed AMR is robust for different images, we implement AMRWT/PW on the BSDS500. Fig. 15 shows the comparison of segmentation results using different algorithms, i.e., SobelAMRWT/PW, gPbAMRWT/PW, OEFAMRWT/PW, and SEAMRWT/PW. The segmentation results demonstrate the effectiveness of AMR for the filtering of useless seeds, Moreover, AMR is effective for both WT and PW.
To compare the performance of different algorithms on the BSDS500, Table II shows experimental results of three evaluation metrics: PRI, CV, and VI. We can see that AMR is more efficient for improving segmentation results obtained by WT or PW compared to MR. MR is sensitive to while AMR is insensitive to . Although MMGMRWT/PW is effective for the oversegmentation reduction by introducing the parameter , segmentation results are sensitive to both and . The gPbAMRWT/PW, OEFAMRWT/PW, and SEAMRWT/PW obtain better performance than SobleAMRWT/PW since the former provides better gradient images. The SEAMRWT/PW obtains the best performance. In addition, AMRWT obtains higher PRI, CV, and lower VI than AMRPW in the same situation.
Because AMR converges quickly, AMR has a high computation efficiency for gradient reconstruction. Table III shows the comparison of running time of AMRWT on different gradient images obtained by Sobel, gPb, OEF, and SE, respectively. We only present the running time of AMRWT here because AMRPW has a similar running time as AMRWT. It can be seen from Table III that AMRWT has a short running time to achieve image segmentation on the BSDS500. The SEAMRWT requires the shortest running time because the corresponding gradient image converges quicker under AMR. Tables IIIII show AMR is effective and efficient for improving seeded segmentation algorithms such as WT and PW.
Additional evidence of the superiority of AMR can be found in Fig. 16 which shows experimental results on images with rich texture and faded boundaries. According to Figs. 1516, we can see that the proposed AMR is effective for different kinds of images.
IvB Seedbased Spectral Segmentation
In this section, we directly construct the affinity matrix on a presegmentation image provided by AMRWT to reduce the size of the affinity matrix, and then compute the subsequent steps of spectral segmentation (we name it AMRSC). Note that we employ AMRWT rather than AMRPW because the former is able to provide better presegmentation results than the latter as shown in Table II. As the presegmentation image only consists of dozens of regions, we consider color feature in CIELAB color space and Gaussian function as the criterion to measure the similarity of two regions. Throughout the paper, we use . It is clear that the affinity matrix produced by AMR is a small matrix. Therefore, the clusters can be detected easily and fast with the means algorithm.
In this paper, the presegmentation depends on AMR. According to Table II, we set and , and we set the number of clusters for the means according to [39], [55]. The proposed AMRSC is evaluated on BSDS500 and compared with algorithms such as gPbowtucm, FNCut, GLgraph, SCG and MCG. Figs. 1718 show that the proposed AMRSC generates better segmentation results than those comparative algorithms. The result demonstrates that AMR is useful for improving spectral segmentation due to two reasons. The first is that the regional spatial information of an image provided by presegmentation is integrated into spectral segmentation, and the second is that the affinity graph is reduced efficiently by removing useless seeds.
Furthermore, we employ the three measures: PRI, CV and VI to compare the proposed AMRSC with nine stateoftheart image segmentation algorithms. Table VI shows the region benchmarks on the BSDS500. In Table VI, the proposed AMRSC clearly dominates other algorithms on PRI and CV, and is on par with SCG on VI mainly due to accurate presegmentation provided by AMRWT. The OEFAMRSC and SEAMRSC provide better performance than gPbAMRSC and SobelAMRSC because OEF and SE obtain better gradient images than gPb and Sobel. In addition, AMRSC is insensitive to the parameter .
We tested the running time complexity on the BSDS500 dataset. The running time comparison is shown in Table IV. On average, generating a presegmentation result with SEAMRWT takes 0.54 seconds (SE generates a gradient image requiring 0.06 seconds. AMRWT takes 0.48 seconds, and ), and constructing an affinity graph and spectral clustering take 0.059 seconds. Consequently, SEAMRSC takes about 0.60 second to segment an image from the BSDS500. In contrast, the gPbowtucm takes almost 106.38 seconds, FNCut takes about 10.58 seconds. As GLgraph has four steps, i.e., oversegmentation, feature extraction, bipartite graph construction and graph partition using spectral clustering, it is more complex than SEAMRSC, and takes almost 7.41 seconds. MCG takes about 18.60s per image to compute the multiscale hierarchy but SCG takes only 2.21s per image. It is clear that our SEAMRSC is the fastest because AMRSC only depends on the gradient information, and the generated affinity matrix is small.
IvC Discussion
AMR has two parameters, and . relates to the convergent condition. Generally, a large value of means a few iterations (a small , where is the number of iterations) while a small value of corresponds to many iterations (a large ). Table V shows the influence of on for test images. We can see that increases with the decrease of but is unchanged when .
Furthermore, to show the influence of on AMR, we implement AMR on the BSDS500 by setting different values of , and Tables VIVII show the results. It is clear that the number of iterations for AMRWT is smaller and running time is shorter if the value of is larger. However, the number of iterations and running time are unchanged when . Therefore, in practical application, users can select different values of according to their requirements.
Furthermore, we implemented SEAMRWT on BSDS500 with different values of . The performance indices of segmentations are shown in Table VIII. By comparing Tables VVIII, we can see that the average number of iterations, running time, and segmentation accuracy are unchanged for AMRWT when . Therefore, the proposed AMR is insensitive to .
The value of controls the initial gradient value of images. A large will cause the contour offset while a small value of will cause too many unexpected small regions. Therefore, we choose and for the BSDS500 in Table IV. To further show the influence of on AMR, Table IX shows the performance indices of segmentations on BSDS500 by setting different values of . It can be seen from Table IX that SEAMRWT is insensitive to if .
V Conclusion
In this work, we have studied the advantages and disadvantages of MR on seeded segmentation algorithms. We proposed an efficient AMR algorithm that can preferably improve seeded segmentation algorithms. The proposed AMR has two significant properties, the monotonic increasingness and the convergence. The monotonic increasingness helps AMR to achieve a hierarchical segmentation. The convergence is able to alleviate the drawback of MR for the filtering of useless regional minima in a gradient image, and guarantees a convergent result. Moreover, we have explored the applications of AMR and have found that AMR is not only able to improve seeded image segmentation results, but also can obtain better spectral segmentation results than stateoftheart algorithms. Furthermore, the proposed AMRSC is computationally efficient because a small affinity matrix is used for spectral clustering. Experimental results clearly demonstrate that the proposed AMRWT generates satisfactory and convergent segmentation results without hardtuning parameters, and the AMRSC outperforms most of the stateoftheart algorithms for image segmentation, and it performs the best in two metrics: PRI and CV.
The segmentation results generated by AMRWT or AMRSC can be directly used in object recognition and scene labeling. However, AMRWT or AMRSC cannot obtain semantic segmentation results compared to the popular convolutional neural network (CNN), e.g., fully convolutional network (FCN) [56]. To further improve the contour quality of segmentation results, traditional algorithms such as conditional random field [57], image superpixel [58], and spatial pyramid pooling [59], are used to improve the performance of CNN on image segmentation. AMR can be also used in CNN to improve semantic segmentation results. For our future work, we plan to investigate how to combine AMR and FCN effectively and efficiently.
Appendix A Proof of
Proof:
Since
we have,
and
According to , in (1), we get
Thus,
In terms of the duality of morphological operation,
Appendix B Proof of theorem 1
Proof:
Let , from Definition 1, we have
Because ,
i.e.,
Appendix C Proof of theorem 2
Proof:
From Definition 1, we have
Since and from Appendix A, we get
We have known that , thus
i.e.,
References
 [1] L. Vincent, “Morphological grayscale reconstruction in image analysis: applications and efficient algorithms,” IEEE Trans. Image Process., vol. 2, no. 2, pp. 176201, Apr. 1993.
 [2] J. Shi, Y. Wang, D. Xu, C. Yan, T. Chen, Y. He, L. Tang, M. Nie, P. Duan, D. Yan, H. Feng, and J. Yao, “Terahertz imaging based on morphological reconstruction,” IEEE J. Sel. Topics Quantum Electron., vol. 23, no. 4, pp. 17, JulyAug. 2017.
 [3] S. Roychowdhury, D. D. Koozekanani, and K. K. Parhi, “Iterative vessel segmentation of fundus images,” IEEE Trans. Biomed. Eng., vol. 62, no. 7, pp. 17381749, Jul. 2015.
 [4] W. Liao, R. Bellens, A. Pizurica, W. Philips, and Y. Pi, “Classification of hyperspectral data over urban areas using directional morphological profiles and semisupervised feature extraction,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 5, no. 4, pp. 11771190, Aug. 2012.

[5]
Z. Huang, X. Wang, J. Wang, W. Liu, and J. Wang, “Weaklysupervised semantic
segmentation network with deep seeded region growing,” in
Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR)
, Jun. 2018, pp. 70147023.  [6] W. Casaca, L. G. Nonato, and G. Taubin, “Laplacian coordinates for seeded image segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2014, pp. 384391.
 [7] L. Vincent and P. Soille, “Watersheds in digital spaces: an efficient algorithm based on immersion simulations,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 13, no. 6, pp. 583598, Jun. 1991.
 [8] C. Couprie, L. Grady, L. Najman, and H. Talbot, “Power watershed: A unifying graphbased optimization framework,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 13841399, Jul. 2011.
 [9] E. O. Rodrigues, L. Torok, P. Liatsis, J. Viterbo, and A. Conci, “KMS: a novel clustering algorithm based on morphological reconstruction,” Pattern Recognit., vol. 66, pp. 392403, Jun. 2017.
 [10] T. Lei, X. Jia, Y. Zhang, L. He, H. Meng, and A. K. Nandi, “Significantly fast and robust fuzzy cmeans clustering algorithm based on morphological reconstruction and membership filtering,” IEEE Trans. Fuzzy Syst., vol. 26, no. 5, pp. 30273041, Oct. 2018.
 [11] T. Lei, Y. Zhang, Y. Wang, S. Liu, and Z. Guo, “A conditionally invariant mathematical morphological framework for color images,” Inf. Sci., vol. 387, pp. 3452, May 2017.
 [12] J. Cheng and J. C. Rajapakse, “Segmentation of clustered nuclei with shape markers and marking function,” IEEE Trans. Biomed. Eng., vol. 56, no. 3, pp. 741748, Mar. 2009.
 [13] B. Peng, L. Zhang, and D. Zhang, “Automatic image segmentation by dynamic region merging,” IEEE Trans. Image Process., vol. 20, no. 12, pp. 35923605, Dec. 2011.
 [14] M. Bosch, C. M. Gifford, A. G. Dress, C. W. Lau, J. G. Skibo, and G. A. Christie, “Improved image segmentation via cost minimization of multiple hypotheses,” arXiv preprint arXiv:1802.00088, 2018.
 [15] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, “Contour detection and hierarchical image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 5, pp. 898916, May 2011.
 [16] P. Dollar and C. L. Zitnick, “Fast edge detection using structured forests,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 37, no. 8, pp. 15581570, Aug. 2015.
 [17] S. Hallman and C. C. Fowlkes, “Oriented edge forests for boundary detection,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 17321740.
 [18] X. Fu, C. Wang, C. Chen, C. Wang, and C.C. Jay Kuo, “Robust image segmentation using contourguided color palettes,” in Proc. IEEE Conf. Int. Comput. Vis. (ICCV), Dec. 2015, pp. 1618–1625.
 [19] D. Comaniciu and P. Meer, “Mean shift: a robust approach toward feature space analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 5, pp. 603619, May 2002.

[20]
A. Y. Ng, M. I. Jordan, and Y. Weiss, “On spectral clustering: Analysis and an algorithm,” in
Proc. Adv. Neural Inf. Proc. Syst. (NIPS), Dec. 2002, pp. 849856.  [21] J. Serra, Image Analysis and Mathematical Morphology, Part II: Theoretical Advances. London: Academic Press, 1988.
 [22] M. E. Plissiti, C. Nikou, and A. Charchanti, “Automated detection of cell nuclei in pap smear images using morphological reconstruction and clustering,” IEEE Trans. Inf. Technol. Biomed., vol. 15, no. 2, pp. 233241, Mar. 2011.
 [23] P. Soille and M. Pesaresi, “Advances in mathematical morphology applied to geoscience and remote sensing,” IEEE Trans. Geosci. Remote Sens., vol. 40, no. 9, pp. 20422055, Sept. 2002.
 [24] W. Liao, M. D. Mura, J. Chanussot, R. Bellens, and W. Philips, “Morphological attribute profiles with partial reconstruction,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 3, pp. 17381756, Mar. 2016.
 [25] P. Soille, Morphological image analysis: principles and applications. Springer Science & Business Media, 2013.
 [26] D. Wang, “A multiscale gradient algorithm for image segmentation using watershelds,” Pattern Recognit., vol. 30, no. 12, pp. 20432052, Dec. 1997.
 [27] M. S. Miri and A. Mahloojifar, “Retinal image analysis using curvelet transform and multistructure elements morphology by reconstruction,” IEEE Trans. Biomed. Eng., vol. 58, no. 5, pp. 11831192, May 2011.
 [28] Y. Li, M. Xu, X. Liang, and W. Huang, “Application of bandwidth EMD and adaptive multiscale morphology analysis for incipient fault diagnosis of rolling bearings,” IEEE Trans. Ind. Electron., vol. 64, no. 8, pp. 65066517, Aug. 2017.
 [29] Y. Li, M. J. Zuo, J. Lin, and J. Liu, “Fault detection method for railway wheel flat using an adaptive multiscale morphological filter,” Mech. Syst. Signal Process., vol. 84, pp. 642658, Feb. 2017.
 [30] J. Chen, C. Su, W. E. L. Grimson, J. Liu, and D. Shiue, “Object segmentation of database images by dual multiscale morphological reconstructions and retrieval applications,” IEEE Trans. Image Process., vol. 21, no. 2, pp. 828843, Feb. 2012.
 [31] H. Shih and E. Liu, “Automatic reference color selection for adaptive mathematical morphology and application in image segmentation,” IEEE Trans. Image Process., vol. 25, no. 10, pp. 46654676, Oct. 2016.
 [32] Y. Teng, Y. Zhang, Y. Chen, and C. Ti, “Adaptive morphological filtering method for structural fusion restoration of hyperspectral images,” IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 9, no. 2, pp. 655667, Feb. 2016.

[33]
Y. Boykov and G. FunkaLea, “Graph cuts and efficient nd image segmentation,”
Int. J. Comput. Vision
, vol. 70, no. 2, pp. 109131, Nov. 2006.  [34] L. Grady, “Random walks for image segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 11, pp. 17681783, Nov. 2006.
 [35] M. Huang, W. Yang, Y. Wu, J. Jiang, W. Chen, and Q. Feng, “Brain tumor segmentation based on local independent projectionbased classification,” IEEE Trans. Biomed. Eng., vol. 61, no. 10, pp. 26332645, Oct. 2014.
 [36] L. Najman and M. Schmitt, “Geodesic saliency of watershed contours and hierarchical segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 12, pp. 11631173, Dec. 1996.
 [37] V. Grau, A. U. J. Mewes, M. Alcaniz, R. Kikinis, and S. K. Warfield, “Improved watershed transform for medical image segmentation using prior information,” IEEE Trans. Med. Imag., vol. 23, no. 4, pp. 447458, Apr. 2004.
 [38] A. M. Mendonca and A. Campilho, “Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction,” IEEE Trans. Med. Imag., vol. 25, no. 9, pp. 12001213, Sept. 2006.
 [39] T. H. Kim, K. M. Lee, and S. U. Lee, “Learning full pairwise affinities for spectral segmentation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 35, no. 7, pp. 16901703, Jul. 2013.
 [40] F. Galasso, M. Keuper, T. Brox, and B. Schiele, “Spectral graph reduction for efficient image and streaming video segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2014, pp. 4956.
 [41] C. Yang, L. Bruzzone, H. Zhao, Y. Tan, and R. Guan, “Superpixelbased unsupervised band selection for classification of hyperspectral images,” IEEE Trans. Geosci. Remote Sens., vol. 56, no. 12, pp. 72307245, Dec. 2018.
 [42] P. Theologou, I. Pratikakis, and T. Theoharis, “Unsupervised spectral mesh segmentation driven by heterogeneous graphs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 2, pp. 397410, Feb. 2017.
 [43] R. Achanta, A. Shaji, K. Smith, A. Lucchi, P. Fua, and S. Süsstrunk, “Slic superpixels compared to stateoftheart superpixel methods,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 34, no. 11, pp. 22742282, Nov. 2012.
 [44] J. Chen, Z. Li, and B. Huang, “Linear spectral clustering superpixel,” IEEE Trans. Image Process., vol. 26, no. 7, pp. 33173330, Jul. 2017.
 [45] X. Wei, Q. Yang, Y. Gong, N. Ahuja, and M. Yang, “Superpixel Hierarchy,” IEEE Trans. Image Process., vol. 27, no. 10, pp. 48384849, Oct. 2018.
 [46] Z. Zhang, F. Xing, H. Wang, Y. Yan, Y. Huang, X. Shi, and L. Yang, “Revisiting graph construction for fast image segmentation,” Pattern Rcognit., vol. 78, pp. 344357, Jun. 2018.
 [47] Y. Xu, E. Carlinet, T. Géraud, and L. Najman, “Hierarchical segmentation using treebased shape spaces,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 3, pp. 457469, Mar. 2017.
 [48] J. Syu, S. Wang, and L. Wang, “Hierarchical image segmentation based on iterative contraction and merging,” IEEE Trans. Image Process., vol. 26, no. 5, pp. 22462260, May 2017.
 [49] J. PontTuset, P. Arbelaez, J. T. Barron, F. Marques, and J. Malik, “Multiscale combinatorial grouping for image segmentation and object proposal generation,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 39, no. 1, pp. 128140, Jan. 2017.
 [50] Y. Chen, D. Dai, J. PontTuset, and L. V. Gool, “Scaleaware alignment of hierarchical image segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.(CVPR), Jun. 2016, pp. 364372.
 [51] F. Zohrizadeh, M. Kheirandishfard, and F. Kamangar, “Image segmentation using sparse subset selection,” in Proc. IEEE Winter Conf. Appl. Comput. Vis.(WACV), Mar. 2018, pp. 14701479.
 [52] T. Cour, F. Benezit, and J. Shi, “Spectral segmentation with multiscale graph decomposition,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Sept. 2005, pp. 11241131.
 [53] D. Hoiem, A. A. Efros, and M. Hebert, “Recovering occlusion boundaries from an image,” Int. J. Comput. Vision, vol. 91, no. 3, pp. 328346, Feb. 2011.
 [54] S. Kim, C. D. Yoo, S. Nowozin, and P. Kohli, “Image segmentation using higherorder correlation clustering,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 9, pp. 17611774, Sept. 2014.
 [55] X. Wang, Y. Tang, S. Masnou, and L. Chen, “A global/local affinity graph for image segmentation,” IEEE Trans. Image Process., vol. 24, no. 4, pp. 13991411, Apr. 2015.
 [56] J. Long, E. Shelhamer, and T. Darrell, “Fully convolutional networks for semantic segmentation,” in Proc. IEEE Conf. Comput. Vis. Pattern Recognit. (CVPR), Jun. 2015, pp. 34313440.

[57]
S. Zheng, S. Jayasumana, B. RomeraParedes, V. Vineet, Z. Su, D. Du, C. Huang, and P. H. S. Torr, “Conditional random fields as recurrent neural networks,” in
Proc. IEEE Intl. Conf. Comput. Vis. (ICCV), Dec. 2015, pp. 15291537.  [58] R. Gadde, V. Jampani, M. Kiefel, D. Kappler, and P. V. Gehler, “Superpixel convolutional networks using bilateral inceptions,” in Proc. Eur. Conf. Comput. Vis. (ECCV), Oct. 2016, pp. 597613.
 [59] L. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and A. L. Yuille, “Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 40, no. 4, pp. 834848, Apr. 2018.
Comments
There are no comments yet.