A skin lesion can be defined as a superficial growth or patch of the skin that does not resemble the area surrounding it. According to the skin cancer foundation, in the past decade (2008-2018), the number of new melanoma cases diagnosed annually increased by 53 percent and one person dies because of melanoma every hour in the world . So, early detection of the skin lesion is one of the foremost challenges in the field of medical engineering. Microscopic (using the microscope) and non-microscopic images, using conventional cameras or smart phones’, as shown in Fig 1, are two types of Medical imaging that are in vivo methods. Additionally, skin lesion segmentation is a major step for malignant/benign classification in which doctors tend to recognize some features; lesion asymmetry, border irregularity, area and the timely progress of moles. Automated methods challenge due to the wide variety of colors, shapes, sizes, in addition to capturing perturbations, like light reflection, bad illumination, hair and others , see Fig 1 (c) and (d).
This work is to segment lesions from 200 images taken from the data base of ISIC 2017 challenge. The segmented lesions are compared with ground truth images and the Jaccard (Intersection over Union or IOU) is calculated. In , color and texture features were extracted from segmented lesion areas. After that, SVM along with k-NN were used for classifications with 61% F1 score. In , thresholding + active contour based method was used with DB1: 10.82% (XOR); DB2: 13.92% (XOR). In , Hill-climbing algorithm + thresholding were used and that had 94.25% (TP). A cutting-edge method has been used by 
, where they proposed a Dense Fully-Convolutional Neural Network (DFCN). By avoiding the redundant computation of neighboring running windows using a dense pooling, they achieved high dice scores in reasonable time.
This report presents two different methodologies for automatic skin lesion segmentation depending on different color spaces and is organized as follows: Section 2 briefly shows different steps for all three methods with explanatory flowcharts. Section 3 is dedicated for results and conclusion.
As mentioned before, this work included two automatic segmentation methods:
Method 1 used watershed with RGB color space as the main segmentation algorithm.
Method 2 used meanshift (clustering) and binarization with gray scale images.
The overall pipeline of this paper is shown in Fig. 2.
In the next three subsections, a walk-through explanation for the two methods is provided.
2.1 Method 1: Watershed
The pre-processing step is a crucial aspect for the effective analysis of pigmented skin lesions. The main purpose of this step is to improve the quality of lesion region by removing unrelated and surplus parts in the background for further processing. Hair removal from skin lesion images is one of the key problems for the precise segmentation and analysis of the skin lesions. To remove the hair, the block diagram as shown in Fig. 3 (a) was used.
Firstly, median filter was used separately on each color dimension (R,G,B) to remove discontented small particle noise like salt-and-pepper noise. Then, the filtered image was used for creating hair mask to inpaint. The block diagram for creating the mask is shown in Fig. 3 (b). In Fig. 3
(b), white top hat has been used with disk (20) type structuring element that can identify hairs. Then binary morphological operations, e.g. closing with a structuring element having size 20 and disk shape, dilation was used to get exact binary mask. To remove the hair, partial differential equation (PDE) and Fast Marching (FM) based inpainting method. After carrying out many experiments, 20 neighboring pixels was used for inpainting. Those two methods are well known as NS and TELEA respectively. Another important aspect of this pre-processing pipeline is that it also reduces some other noise, i.e. reflections, shadows, skin lines and air bubbles from the image.
For the segmentation, Watershed was used and its pipeline is shown in Fig. 3 (c). To overcome the over segmentation marker-controlled, watershed segmentation has been used . After OTSU thresholding, some morphological operation, e.g. opening, and closing, have been done to reduce some white and black noise respectively. To extract the exact sure foreground, distance transform was used that provided a metric or measure of the separation of points in the image. Then marker was created which was used to segment skin lesion using watershed algorithm. Markers based watershed segmentation showed very good results.
2.2 Method 2: Mean-shift
Fig 4 provides the bird’s-eye view of the methodology used for Method 2.
BGR to Gray scale: After some experiments, gray scale gave good results in comparison with HSV (H and V channels). Additionally, Gray scale processing provided close results to that given by BGR images.
Border filling: The aim of this process is to overcome the effect of the dark borders, in microscopic images especially, on the thresholding decision. During this process, binary morphological dilation is applied to remove the one-pixel tiling. Then, a morphology opening is used to block routes leading to the heart of the image (non-border pixels). After that, region growing is used with four single seeds in the four corners of the image, the goal is to fill pixels with intensities less than a threshold with a pre-defined bright intensity.
Hair removal: Hair is one of the artifacts that could be found in a lesion image. To solve this issue, morphology top-hat is used to create the hair mask, then this mask is passed to the inpainting TELEA algorithm along with the gray image to fill hair region defined by the mask with intensities derived from neighborhoods. enditemize
2.2.2 Mean shift and OTSU thresholding
After Gray-to-BGR transformation is applied (replication is done behind the scene), the result is provided to the pyramids mean shift algorithm which tries to merge pixels close to each other in distance and color spaces. The distance bandwidth and the color bandwidth were tuned to get the required lesion in many cases. The result of mean-shift is passed to OTSU thresholding, which gives the first unprocessed result which may have some exteriors and gaps in some cases.
2.2.3 Post morphology
To ameliorate the problem of exteriors, morphology closing then opening was used to enhance the boundaries of the lesion and remove impairments.
3 Results and conclusion
To evaluate the performance for each method, as mentioned earlier, IoU metric was used. This has been done by counting pixels in intersection region between the result binary image and the provided ground truth and divide it by the number of pixels in the union region. Overall, results in this paper are divided into three subsections. Subsection 3.1 is dedicated for Method 1 results, subsection 3.2 for Method 2, and subsection 3.3 is for comparison between method 1 and method 2.
3.1 Method 1 results
The First step, pre-processing, was to reduce noise from the skin image. The outcome is given in Table 1. As can be seen from that table, proposed pipeline not only removes the hair but also reduces ruler noise as well. Another noticeable aspect is if there is no hair nor noise then inpainting keeps this image as it is. After removing hair, marker-controlled watershed was used to segment the ROI of the skin lesion. Some of the segmented lesions by the proposed pipeline are given in Table 2.
From Table 2, in most of the cases, the segmentation was almost same as ground truth. In some cases especially for the microscopic image performance metric become very less. But over the 200 images, the average Jaccard Index for proposed pipeline are 89.16 % and 89.08 % for NS based and TELEA based inpainting in pre-processing respectively.
3.2 Method 2 results
This method was applied on BGR images, as well as on gray images. With BGR, an average of 76.94%, whilst with gray images, it gave a maximum of 74.82%. This perhaps is a significant outcome, telling that RGB color space carries more information than that can gray images carry. In Table 3, it is clearly visible that BGR results are of higher Jaccard, however, it requires more computational resources.
3.3 Comparing both methods and Conclusion
After extensive experiments, Method 1 showed noticeably higher Jaccards than Method 2. Table 4 shows 4 different examples. The first row is an ordinary one with the lesion clearly recognized by both methods. The Second row shows a hairy image. Here, Method 2 outperforms method 1 due to the perfect hair removal techniques used. Row 3 is to show a microscopic example, that is, to see the effect of using Border filling, see Section 2.2.1. Lastly, row 4 shows a lesion with varying intensities. In the end, both methods were able to distinguish the vast majority of the 200 ISIC-2017 lesions with pretty good performances.
|Original||Ground truth||method 1||method 2|
-  Skin Cancer Facts & Statistics. Access: 22-May-2018. Available at: www.skincancer.org/skin-cancer-information/skin-cancer-facts.
-  E. Nasr-Esfahani, S. Rafiei, M. H. Jafari, N. Karimi, J. S. Wrobel, K. Najarian, S. Samavi, S.M. Reza Soroushmehr, “Dense Fully Convolutional Network for Skin Lesion Segmentation,”arXiv:1712.10207.
-  B. S. Lin, K. Michael, S. Kalra, H.R. Tizhoosh, “Skin Lesion Segmentation: U-Nets versus Clustering,” IEEE Symposium Series on Computational Intelligence (SSCI), 2017 , February 2018.
-  S. Ra, M. Suhilb, and D. S. Guru, “Segmentation and Classification of Skin Lesions for Disease Diagnosis,” International Conference on Advanced Computing Technologies and Applications, vol. 45, pp. 76-85, 2015.
-  Z. Ma, J. Manuel, and R. S. Tavares, “A Novel Approach to Segment Skin Lesions in Dermoscopic Images Based on a Deformable Model,” IEEE Journal of Biomedical and Health Informatics, vol. 20, no. 2, March 2016.
-  Q. Abbas, I. F. Garcia, M. E. Celebi, W. Ahmad, and Q. Mushtaq, “A perceptually oriented method for contrast enhancement and segmentation of dermoscopy images,” Wiley Online Library, pp. 490-497, August 2012.
A. Telea, “An Image Inpainting Technique Based on the Fast-Marching Method,”J. Graphics Tools, vol. 9, no. 1, pp. 23-24, 2004.
-  K. L. Mon, “Automatic Image Segmentation Using Edge and Marker-Controlled Watershed Transformation,” International Conference on Advances in Engineering and Technology (ICAET), March, 2014.