Catheter-based Intravascular Ultrasound (IVUS) has captured considerable attention in the last two decades. This worldwide attention is mostly due to the ability of the imaging method to picture the inside of the human coronary arteries and, hence, provide an opportunity to diagnose and treat cardiovascular diseases such as atherosclerosis (e.g., thin-cap fibroatheroma) that causes a heart attack and a brain stroke frostegaard2005sle . Aside from this, the IVUS technique can be helpful in visualizing some internal structures of the human coronary such as the lumen, and thickness and distribution of the plaques katouzian2012state . Therefore, IVUS is regularly used to locate the atherosclerosis lesions in the coronary arteries to study the lumen and plaque dimensions, and to guide intervention and stent deployment ma2016review .
A typical IVUS imaging system consists of four parts: catheter, transducer, pullback device, and scanning console. The catheter is composed of a 150 cm long guidewire and a tip of 1.2-1.5 mm in size. It is usually inserted in the femoral artery and proceeds toward the coronary arteries. The catheter is responsible for carrying the ultrasound transducer, or other necessary devices, such as inflatable balloons and stents katouzian2012state . The transducer is a miniaturized ultrasound probe that emits ultrasound pulses and listens for the backscattered signal. After the catheter has reached the distal end of the coronary, it needs to be manually or automatically pulled back. The speed of the pullback varies between 0.5-1 mm/s katouzian2012state . The scanning console is essentially a computer used to post-process the acquired signals (using amplification, filtering, etc.) to provide a user-friendly environment for the surgeon to control the device.
Segmentation of the acquired IVUS images is among the most challenging tasks in medical image analysis. In particular, delineating the interior (lumen) and exterior (media) vessel walls is problematic due to the presence of various artifacts such as motion of the catheter after a heart contraction, guide wire effects, bifurcation and side-branches or similar echogenicity between the vessel wall and some plaques. In some cases even the difference in transducer frequencies affect the segmentation results katouzian2012state .
The intrinsic difficulty of IVUS segmentation has attracted many researchers to study and develop solutions using different methodologies, such as intensity-based, statistics and probability-based, active contour and graph search-based approaches. In addition, several methods have been proposed to segment either the lumen or the media or both. A great number of approaches in the literature have utilized the 2D information provided as cross-sectional frames to segment the lumen and media. These 2D cross-sectional gray-scale images are formed after digitization of the backscattered RF signals and are called IVUS B-mode frames. To the best of our knowledge, recent approaches have mostly worked on the B-mode frames which will be reviewed in the following paragraph. For more in-depth reviews of the methods published before 2013, please refer tokatouzian2012state ; balocco2014standardized .
The lumen and media-adventitia border variations have been modeled within a shape space in unal2008shape . The lumen segmentation is then performed by maximizing a nonparametric probability density energy. Also, the edge information has been used to segment the media-adventitia. A physics-based model of the IVUS signal scattered by the structure of the vessel has been used in mendizabal2016physics
to estimate the differential backscattering cross-sections from the IVUS RF signal. The segmentation curve is obtained after training a Support Vector Machine (SVM) model using the annotated data. Deformable models have been used intaki2008automatic ; zhu2011snake ; mendizabal2013segmentation to detect the border of lumen/intima and media/adventitia. In taki2008automatic anisotropic diffusion followed by an edge detector are used to create an initial segmentation which is then corrected using both geometric and parametric deformable models. In zhu2011snake an improved version of gradient vector flow (iGVF) has been proposed which includes a balloon force in the snake model that lets the contour pass over leaks and bifurcations. A probabilistic method that formulates the deformation of a lumen contour curve and can be minimized has been proposed in mendizabal2013segmentation . However, every first frame of the sequence needs user interactions to manually segment the lumen and media. Following this, a SVM is trained over the annotated data to compute the probability that each pixel is blood or non-blood. A fourfold algorithm based on a deterministic statistical strategy for segmenting the media has been proposed in zakeri2017automatic . Their method consists of preprocessing, initial contour detection, active contour segmentation, and contour refinement. First, a sparse binary image is constructed using the local appearance model and the initial contour is elicited. To achieve this, a feature vector is built for each pixel. It includes gray-level values of the pixel’s neighbours, the average intensity of neighbours and the gray-level values of the pixel’s neighbours in a contour-enhanced version of the imagezakeri2017automatic
. The K-SVD method is utilized to classify the extracted feature vector. The initial contour is then refined. Next, an active contour model is used to delineate the media border in polar coordinates. The detected contours are then refined using the information provided by identifying the calcification and shadow regions. Artificial Neural Networks have been employed insu2016artificial to represent the spatial and neighbouring features of the IVUS image data. As a result, two different vascular structures for lumen and media are extracted and optimized using two ANNs. The borders obtained are then refined and smoothed by an active contour model. In yan2017novel the lumen is segmented by a combination of image gradient and fuzzy connectedness model and the media-adventitia border is extracted by a fast marching model. A sequential forward selection process using SVMs and PR curve has been employed to conduct an in-depth analysis of several image features in vercio2016assessment . It has been shown that the median filtered image and Haralick’s texture features haralick1979statistical provide stronger discrimination capabilities for arterial structures. A limitation of their analysis is that it only works for artifact-free IVUS sequences.
As we can see, most approaches have employed either a type of energy minimization method or require annotated data in order to train a classifier or an ANN. However, in this paper we propose a straightforward approach that not only does not require training but also does not use any variational method or deformable model. We show that by extracting EREL features faraji2015erel ; faraji2015extremal the problem of the IVUS segmentation can be relaxed to a region selection. In particular, we illustrate that it is very likely to find regions similar to lumen and media among the extracted ERELs. Therefore, we propose a selection procedure that efficiently chooses ERELs that are most similar to lumen and media.
The rest of this paper is organized as follows. In Section 2 we present our proposed method and describe the sequence of Intravascular Ultrasound images that we use throughout this paper. Section 3 illustrates the segmentation results of our proposed method. In Section 4 we discuss the advantages and weaknesses of the proposed method. Finally, the concluding remarks are given in Section 5.
2 Materials and Method
Our proposed method has been evaluated on the test set of publicly available dataset consisting of 326 in-vivo pullbacks of the human coronary artery frames that were acquired by the Si5 (Volcano Corporation), equipped with a 20 MHz Eagle Eye monorail catheter balocco2014standardized . The dataset includes a multi-frame 3D context that has between 20 to 50 gated frames acquired using a full pullback at the end-diastolic cardiac phase from 10 patients. Manual annotations for IVUS images are available in the dataset. The annotations have been provided by four clinical experts who work regularly with IVUS echograph. The experts were not aware of other expert’s manual annotations and two of them repeated the annotations after about one week from their first delineation balocco2014standardized .
The test set contains several types of common artifacts. Specifically, it includes 44 images containing bifurcation, 93 images with a side vessel artifact, and 96 images that have been contaminated by a shadow artifact (some images contain more than one artifact). There are also 143 images that do not contain any serious artifacts except for plaque.
2.2 Proposed Method
In this paper we present a segmentation approach for 20 MHz Intravascular Ultrasound images based on a region detection strategy. Particularly, we investigate whether a recently proposed novel feature extraction method called Extremal Regions of Extremum Levels (EREL)faraji2015erel ; faraji2015extremal can segment the most essential regions of interest (lumen and media) from the IVUS images required to establish the atherosclerotic plaque area destrempes2014segmentation . The proposed method consists of four steps. We first remove the typical artifacts of IVUS frames, such as ring-down effects and calibration squares. Then ERELs are extracted and the obtained regions are filtered based on their types. Next, we perform a region selection procedure to specify two regions as lumen and media. Finally, the contour of the two selected regions is traced and smoothed by an ellipse fitting algorithm.
Most of the IVUS images have been contaminated by speckle noise balocco2014standardized . Speckle is a multiplicative noise that imposes difficulties in processing the Ultrasound images loizou2008despeckle . Therefore, to decrease the sensibility of our method to speckle noise, we first use a non-linear median operator to filter the IVUS images.
One of the main identifiable artifacts in IVUS images is the ring-down effects of the catheter that need to be eliminated from the B-mode frame or from its polar image. Otherwise, there is a high risk of obtaining an erroneous segmentation. To remove the ring-down effects of the catheter we employ the method proposed in unal2008shape which is a very fast and straightforward procedure. Detecting the ring-down artifact can be done by processing the whole volume since the artifact is almost available in all of the IVUS frames. Therefore, taking the minimum over all the frames generates an image where there is a significant contrast between the artifact and the non-artifact pixels.
where is a set of available frames in a particular IVUS sequence. We can then locate the artifacts’ coordinates by subtracting the artifact zones from every frame, as in unal2008shape . Figure 1(b) and Figure 1(f) show the resulted minimum image in B-mode and polar frame respectively.
Another type of artifact that we can detect using Eq.1 is the calibration square artifact. These small squares have a very bright constant intensity in all frames that remains bright in the minimum image. In several longitudinal cuts of the IVUS volume the effects of these artifacts are revealed as horizontal lines. Figure 1 illustrates both the artifacts and the resulting image after removing them.
2.4 Extremal Regions of Extremum Levels
in conjunction with the edge information to detect a series of connected pixels from the image. The edge information of the image is included in the method by using the Maxima of Gradient Magnitude (MGM) points. The idea underlying EREL is to binarize the image with all possible integer thresholds and analyze the results obtained based on their global criterion and their local edge information. The regions belonging to the globally distinguished levels (Extremum Levels) are then extracted from the union-find tree.
Generally, two types of regions can be extracted from a gray-level image. The first type includes regions that evolve from brighter surfaces to darker boundaries which are identified by . The superscript ’’ emphasizes the fact that the intensity values are decreased from the surface of the regions towards the boundaries. The second type consists of regions that evolve from darker surfaces to brighter boundaries and are denoted by . This type of the region is consistent with the inherent characteristics of the lumen and media visualized by the backscattered 20MHz IVUS signals. Therefore, we only need to extract regions to obtain the lumen and media because both regions evolve from darker surfaces to brighter boundaries.
To use EREL, we need to set several initialization parameters, namely , , , . These parameters define the functionality of the detector and can be tuned based on the application faraji2015extremal ; faraji2015erel . In particular, we use , to set the minimum and maximum area of the extracted regions. To better separate small regions from bigger ones, we choose a value for that correlates with image dimensions. Specifically, we set and where and represent the number of rows and columns of the IVUS image, respectively. The parameter is usually in and represents the strength of the resulting interest points faraji2015extremal . In this study we set . Finally, denotes the width of the moving window over the global criterion vector faraji2015extremal . The extracted EREL regions using the above-mentioned values are illustrated in Figure 2.
However, not all four types of the regions depicted in Figure 2 encompass lumen and media regions. In fact, we only need to extract large area regions as illustrated in Figure 2(d). Since large area regions contain the actual lumen and media segments, detectors need not track regions and, therefore, omit unnecessary computations which eventually helps to have a faster detector.
2.5 EREL Selection
The goal of this section is to address the problem of finding the most appropriate ERELs to be designated as lumen and media. By the most appropriate we mean the closest regions to the gold standard. As can be seen in Figure 2(d), although ERELs are nested regions, it is clear that there is at least one EREL that is very close to the true lumen and similarly there is at least one EREL that corresponds to the true media. Therefore, we can relax the problem of lumen and media segmentation to only a selection procedure, i.e., assigning two nested ERELs to lumen and media.
Local maxima searching is the approach that we employ to select lumen and media from the nested set of ERELs. Assume that we have a vector (denoted by ) representing the evolution of the regions. The index of each element of the vector corresponds to a region. Since the extracted ERELs start from the smallest enclosing region and end with the largest enclosing region, the actual region corresponding to the lumen should be found among the regions located at the early indices of the vector. Likewise, a region representing the media is expected to be found among the regions belonging to the end section of the vector .
Our aim is to construct a vector that ultimately gives us the stability of the regions in terms of the length of the boundary, average intensity and entropy variation. As we can see in Figure 2(d), the boundaries of the regions are not smooth and are subject to large variations. Therefore, lumen and media regions should be selected among those regions that have more stable boundary length variations. Also, the average intensity of the regions should be stable enough (i.e., they should not change much over several subsequent regions). Entropy measure can be used in order to create a feature vector that is sensitive to textural information of the regions.
|(i) No Artifact||(ii) Plaque and Shadow|
The calculation of these three features are straightforward. The boundary lengths of the regions are available as an output of the EREL algorithm and are calculated based on a bottom-up tracking of boundary pixels along a parametric curve .
Additionally, the calculation of the average intensity of ERELs can be readily done.
where represents the area of the region and is the intensity value of pixel . The entropy measure of a grayscale region haralick1979statistical is denoted as follows.
where is the value of the bin of the normalized histogram of the region and captures the probability of having a pixel with a certain gray-value. is the number of available bins in the normalized histogram.
Afterwards, for every IVUS image, we create a vector () where each element is obtained from the product of the above-mentioned measures for each region.
where is the number of regions extracted by the EREL algorithm.
Vector illustrates the variation of textural information of regions through the extraction and evolution of regions. It is strictly increasing because regions are nested and non-repetitive. More stable sequences of vector are more likely to represent lumen and media since these stable sequences shows that the extremal regions are subject to saturation and the subsequent regions might contain a noticeable change. The best way to find the stable regions is to create a vector describing the stability score of regions. Every element of the stability vector is calculated as follows.
where specifies a specific element of the vector and varies from one to the number of the detected regions for an IVUS image.
The local maxima of the stability score points to regions with high stability because the ratio of their current value to the change among their two neighbours is larger than the other surrounding elements. So, we select lumen and media from the detected local maxima. Specifically, a region with higher prominence value among the first two peaks is considered as lumen. If the IVUS image contains no artifacts, the media will be represented by the last detected peak. Based on our observation, the stability score of the images that contain serious artifacts have none or a small number of peaks since the presence of the artifacts interferes with the natural extraction of regions (see Figure 3(ii)). Therefore, when a small number of local maxima is detected, we consider the last extracted region as media. This process for an IVUS with no particular artifact is illustrated in Figure 3. The local maxima of the stability score indicates the regions for which the variation of the textural characteristics is more stable than their surrounding regions. As can be seen in Figure 3(i)(a), the second peak is selected as a suitable region for the lumen since it has a higher prominence than the first peak. Also, the region corresponding to the last peak of which is chosen as the media has been shown in Figure 3(i)(b).
where represents the area of the region , is the median of a vector of all region areas, and denotes the Median Absolute Deviation and is calculated by the following equation.
Therefore, the regions that have a modified Z-score less than and greater than are unlikely to represent lumen and media and hence can be removed from the selection process.
2.6 Contour Extraction
Since the general shape of the lumen and media regions of the vessel are very similar to conic sections, we propose to represent lumen and media by ellipses. To find the pixels inside and on the ellipse border, it is sufficient to find the orientation of the ellipse, the major and minor axis length. Generally, EREL outputs all pixels belonging to each extracted region in addition to its shape description parameters which are three coefficients of ellipse equation faraji2015erel ; faraji2015extremal . Specifically, if a region is described by an ellipse with three coefficients, namely , and then:
There is a direct relationship between the parameters of the fitted ellipse and the second central momentsshapiro2001computer :
where , and are the second order central moments and are calculated as follows shapiro2001computer :
where represents the area of the region and specifies the coordinates of the region’s centroid.
After having obtained matrix (reported by EREL), finding pixels belonging to ellipse border of the regions is straightforward. We assume that where
denotes a matrix containing the eigenvectors and
indicates a matrix of eigenvalues. The minimum () and maximum () eigenvalues in are used to calculate the length of the minor() and major () axes of the ellipse, respectively szeliski2010computer .
The orientation of the ellipse () can be recovered by calculating the angle between the major axis and the axis. Let where and are two column vectors corresponding to the minimum and maximum eigenvalues, respectively. The orientation of the ellipse is then obtained as follows.
2.7 Computational Cost
The run time of the proposed method depends only on the complexity of EREL which is faraji2015extremal , where is the total number of pixels in the image. The subsequent operation proposed in this paper in Section 2.5 works on a constant number of regions. The maximum number of nested regions that can be extracted from the same root pixel is at most 256 for an 8-bit image faraji2015extremal . In our experiments with IVUS images, the number of candidate ERELs () rooted from the center of image are even less (lower than 75). Therefore, the region properties denoted in Eq.2, Eq.3, Eq.4 and Eq.6 and the second moment matrix are calculated on a constant number of regions containing far less than pixels. Therefore, the overall run time of the proposed method is in the worst case. A comparison between the actual run time of the proposed method and the methods reported in balocco2014standardized is shown in Table 1.
|Category||Semi/Auto||2D/3D||Time per frame||Hardware used|
|Proposed Method||Lumen and media||AUTO||2D||0.19 s||Core i7-4700HQ, 2.4 GHz|
|Unal et al. unal2008shape||Lumen and media||SEMI||2D||3.25 s||Pentium 6200 2.13 GHz|
|Wang et al.balocco2014standardized||Lumen||SEMI||2D||1 m 40 s||Xeon 2.67 GHz|
|Destrempes et al.balocco2014standardized||Lumen and media||SEMI||2D||8.64 a||Core i7 Q740 @ 1.73 GHz|
|Downe et al. downe2008segmentation||Lumen and media||AUTO||3D||0.16 s||Core 2, 2.4 GHz|
|Alberti et al.balocco2014standardized||Lumen||AUTO||3D||13 s||Core 2, Duo 2.13 GHz|
|Ciompi et al. ciompi2012holimab||Media||AUTO||2D||20 s||Core i7, 2.8 GHz|
|Mendizabal et al. mendizabal2013segmentation||Lumen||SEMI||2D||4.96 s||Core i7, 2 GHz|
|Exarchos et al. balocco2014standardized||Lumen and media||AUTO||2D||0.5 s||Core 2, Duo 3.33 GHz|
In this section, we present the segmentation results of our method. Also, by showing the best case results, we demonstrate that regions very close to the lumen and media exist among the extracted ERELs and a proper selection strategy (the proposed method) can distinctively select the lumen and media regions from the extracted ERELs. Furthermore, we present extensive evaluation results of our method based on three standard evaluation metrics on IVUS frames containing various artifacts.
3.1 Evaluation Measures
To assess the segmentation obtained by our method, we employ three evaluation metrics, namely Jaccard Measure (JM), Hausdorff Distance (HD), and Percentage of Area Difference (PAD). Using these metrics that have been also used inbalocco2014standardized to evaluate the results of 8 state-of-the-art methods on the same dataset makes it possible to draw a fair comparison between our method and the results reported in balocco2014standardized .
The Jaccard Measure is calculated based on the comparison of the automatic segmentation result and the manual segmentation delineated by experts. It quantifies the overlap area between the automatic and manual segmentation as computed by the following equation.
Where is the vessel region segmented by the method and represents the region that has been segmented manually by experts.
The Hausdorff Distance between the automatic () and manual () curves is the greatest distance of all points belonging to to the closest point in and is defined as follows molinari2011caudles .
To calculate first the minimum of all Euclidean distances from each point belonging to to all points in is obtained. Then, is computed by taking the maximum of all the minimum distances. Similarly, is computed by taking the maximum of all minimum distances from to molinari2011caudles .
The Percentage of Area Difference calculates the segmentation area difference between the automatic () and manual () segmentation and is computed as follows.
|General Performance||EREL||0.22 (0.12)||0.91 (0.04)||0.03 (0.03)||0.50 (0.45)||0.83 (0.15)||0.13 (0.15)|
|Intra-obs||0.28 (0.13)||0.88 (0.05)||0.11 (0.08)||0.24 (0.12)||0.92 (0.03)||0.06 (0.04)|
|Inter-obs||0.17 (0.13)||0.93 (0.05)||0.04 (0.06)||0.14 (0.12)||0.95 (0.03)||0.03 (0.03)|
|No Artifact||EREL||0.20 (0.09)||0.92 (0.03)||0.02 (0.02)||0.23 (0.17)||0.92 (0.05)||0.03 (0.05)|
|Intra-obs||0.28 (0.13)||0.88 (0.05)||0.11 (0.08)||0.24 (0.12)||0.92 (0.03)||0.06 (0.04)|
|Inter-obs||0.17 (0.13)||0.93 (0.05)||0.04 (0.06)||0.14 (0.12)||0.95 (0.03)||0.03 (0.03)|
|Bifurcation||EREL||0.36 (0.22)||0.85 (0.07)||0.07 (0.07)||0.95 (0.49)||0.67 (0.16)||0.27 (0.17)|
|Intra-obs||0.30 (0.12)||0.88 (0.04)||0.09 (0.06)||0.24 (0.09)||0.92 (0.02)||0.06 (0.03)|
|Inter-obs||0.18 (0.21)||0.92 (0.07)||0.05 (0.09)||0.15 (0.13)||0.95 (0.04)||0.03 (0.03)|
|Side Vessels||EREL||0.20 (0.09)||0.90 (0.04)||0.03 (0.03)||0.62 (0.53)||0.78 (0.16)||0.18 (0.16)|
|Intra-obs||0.30 (0.13)||0.88 (0.05)||0.10 (0.08)||0.24 (0.11)||0.92 (0.04)||0.06 (0.04)|
|Inter-obs||0.20 (0.11)||0.91 (0.05)||0.06 (0.05)||0.15 (0.10)||0.95 (0.03)||0.03 (0.04)|
|Shadow||EREL||0.22 (0.11)||0.89 (0.04)||0.03 (0.03)||1.10 (0.38)||0.64 (0.13)||0.31 (0.14)|
|Intra-obs||0.31 (0.13)||0.88 (0.05)||0.11 (0.08)||0.27 (0.15)||0.92 (0.04)||0.06 (0.05)|
|Inter-obs||0.18 (0.14)||0.93 (0.05)||0.04 (0.06)||0.14 (0.10)||0.96 (0.03)||0.02 (0.02)|
The best case performance results of the proposed method. Measures represent the mean and standard deviation (std) evaluated on 435 frames of datasetbalocco2014standardized . The measures are categorized based on the presence of a specific artifact in each frame. The evaluation measures are Jaccard Measure (JM), Hausdorff Distance (HD), and Percentage of Area Difference (PAD).
3.2 Best Case Results
In order to show that the extracted EREL regions have the potential to represent lumen and media regions, we evaluate all of the extracted ERELs by calculating the evaluation metrics (Section 3.1) for the contours of each EREL and the manually annotated contours. Then, the ERELs that correspond to the maximum JM of lumen and media are selected as the best extracted ERELs for that frame. The quantitative results in comparison with the intra-observer and inter-observer variability are reported in Table 2.
3.3 EREL Selection Results
Qualitative evaluations are illustrated in Figure 4 and show the successful segmentation results of the proposed EREL selection strategy for 20 IVUS frames. The lumen areas are highlighted by the magenta colour while the media regions are green. Also, the manually annotated contours for both lumen and media are drawn as yellow dashed lines. As we can see, the chosen frames contain a variety of lumen and media morphologies.
A detailed evaluation result and comparison with 9 recently published IVUS segmentation methods are reported in Table 3 where the performance of the proposed EREL selection strategy in the presence of various artifacts is shown as well.
|General Performance||Proposed Method||0.30 (0.20)||0.87 (0.06)||0.08 (0.09)||0.67 (0.54)||0.77 (0.17)||0.19 (0.18)|
|Unal et al. unal2008shape||0.47 (0.39)||0.81 (0.12)||0.14 (0.13)||0.64 (0.48)||0.76 (0.13)||0.21 (0.16)|
|Wang et al.balocco2014standardized||0.51 (0.25)||0.83 (0.08)||0.14 (0.12)||–||–||–|
|Destrempes et al.balocco2014standardized||0.34 (0.14)||0.88 (0.05)||0.06 (0.05)||0.31 (0.12)||0.91 (0.04)||0.05 (0.04)|
|Downe et al. downe2008segmentation||0.47 (0.22)||0.77 (0.09)||0.15 (0.12)||0.76 (0.48)||0.74 (0.17)||0.23 (0.19)|
|Alberti et al.balocco2014standardized||0.46 (0.30)||0.79 (0.08)||0.16 (0.09)||–||–||–|
|Ciompi et al. ciompi2012holimab||–||–||–||0.57 (0.39)||0.84 (0.10)||0.12 (0.12)|
|Mendizabal et al. mendizabal2013segmentation||0.38 (0.26)||0.84 (0.08)||0.11 (0.12)||–||–||–|
|Exarchos et al. balocco2014standardized||0.42 (0.22)||0.81 (0.09)||0.11 (0.11)||0.60 (0.28)||0.79 (0.11)||0.19 (0.19)|
|No Artifact||Proposed Method||0.29 (0.17)||0.88 (0.05)||0.08 (0.07)||0.31 (0.23)||0.89 (0.07)||0.07 (0.08)|
|Lo Vercio et al. vercio2016assessment||–||0.83 (0.05)||0.18 (0.06)||–||–||–|
|Bifurcation||Proposed Method||0.53 (0.34)||0.79 (0.10)||0.15 (0.17)||1.22 (0.45)||0.57 (0.13)||0.32 (0.19)|
|Unal et al. unal2008shape||0.65 (0.47)||0.76 (0.14)||0.18 (0.15)||0.57 (0.49)||0.78 (0.13)||0.19 (0.15)|
|Wang et al.balocco2014standardized||0.54 (0.27)||0.81 (0.11)||0.14 (0.13)||–||–||–|
|Destrempes et al.balocco2014standardized||0.42 (0.18)||0.85 (0.06)||0.08 (0.06)||0.32 (0.13)||0.91 (0.03)||0.06 (0.04)|
|Downe et al. downe2008segmentation||0.64 (0.27)||0.70 (0.11)||0.21 (0.15)||0.79 (0.53)||0.71 (0.19)||0.24 (0.21)|
|Alberti et al.balocco2014standardized||0.61 (0.43)||0.75 (0.10)||0.20 (0.10)||–||–||–|
|Ciompi et al. ciompi2012holimab||–||–||–||0.52 (0.29)||0.85 (0.07)||0.09 (0.07)|
|Mendizabal et al. mendizabal2013segmentation||0.53 (0.36)||0.79 (0.12)||0.17 (0.18)||–||–||–|
|Exarchos et al. balocco2014standardized||0.47 (0.23)||0.80 (0.09)||0.10 (0.09)||0.63 (0.25)||0.78 (0.11)||0.23 (0.23)|
|Side Vessels||Proposed Method||0.24 (0.11)||0.87 (0.05)||0.06 (0.05)||0.74 (0.18)||0.73 (0.60)||0.21 (0.18)|
|Unal et al. unal2008shape||0.51 (0.39)||0.79 (0.12)||0.17 (0.14)||0.57 (0.39)||0.78 (0.11)||0.18 (0.12)|
|Wang et al.balocco2014standardized||0.59 (0.23)||0.80 (0.10)||0.16 (0.13)||–||–||–|
|Destrempes et al.balocco2014standardized||0.36 (0.15)||0.87 (0.04)||0.07 (0.04)||0.31 (0.12)||0.91 (0.04)||0.04 (0.04)|
|Downe et al. downe2008segmentation||0.46 (0.19)||0.77 (0.08)||0.15 (0.11)||0.76 (0.47)||0.74 (0.16)||0.22 (0.20)|
|Alberti et al.balocco2014standardized||0.47 (0.24)||0.79 (0.07)||0.17 (0.09)||–||–||–|
|Ciompi et al. ciompi2012holimab||–||–||–||0.53 (0.37)||0.85 (0.09)||0.10 (0.13)|
|Mendizabal et al. mendizabal2013segmentation||0.38 (0.19)||0.84 (0.07)||0.11 (0.11)||–||–||–|
|Exarchos et al. balocco2014standardized||0.53 (0.24)||0.77(0.09)||0.16 (0.12)||0.63 (0.31)||0.78 (0.12)||0.18 (0.16)|
|Shadow||Proposed Method||0.29 (0.20)||0.86 (0.07)||0.08 (0.09)||1.24 (0.39)||0.58 (0.13)||0.37 (0.15)|
|Unal et al. unal2008shape||0.57 (0.39)||0.78 (0.12)||0.17 (0.12)||0.66 (0.50)||0.77 (0.13)||0.19 (0.15)|
|Wang et al.balocco2014standardized||0.59 (0.27)||0.81 (0.10)||0.18 (0.16)||–||–||–|
|Destrempes et al.balocco2014standardized||0.39 (0.18)||0.87 (0.05)||0.06 (0.05)||0.33 (0.14)||0.92 (0.03)||0.05 (0.04)|
|Downe et al. downe2008segmentation||0.55 (0.26)||0.76 (0.11)||0.14 (0.13)||0.77 (0.48)||0.74 (0.16)||0.22 (0.19)|
|Alberti et al.balocco2014standardized||0.53 (0.29)||0.78 (0.08)||0.18 (0.09)||–||–||–|
|Ciompi et al. ciompi2012holimab||–||–||–||0.58 (0.36)||0.84 (0.09)||0.11 (0.11)|
|Mendizabal et al. mendizabal2013segmentation||0.43 (0.27)||0.83 (0.09)||0.12 (0.11)||–||–||–|
|Exarchos et al. balocco2014standardized||0.46 (0.19)||0.80 (0.10)||0.12 (0.12)||0.57 (0.28)||0.82 (0.11)||0.14 (0.17)|
The present study is twofold. We first investigate whether the extracted ERELs faraji2015erel ; faraji2015extremal are proper regions to be considered as lumen and media delineations of 20 MHz Intravascular Ultrasound frames taken from the inside of the human coronary arteries. We quantitatively (Table 2) validate that very close candidates can be found from the small number (almost less than 50) of extracted ERELs for each IVUS frame. Specifically, the best case results demonstrate the average of the closest extracted EREL to the lumen and media among all 435 images of the dataset balocco2014standardized . As the results of the best case study suggest, if we design a proper selection strategy we are able to find two regions out of the extracted ERELs that are very close to the actual lumen and media structure.
Secondly, we propose an ERELs selection strategy based on creating a vector that represents several textural characteristics of the regions and search for the regions which have higher textural stability scores. This method works very well when no artifact is present in the IVUS image or more accurately, when no major artifact is attached to the media. In this case, the average Hausdorff distance for both lumen and media is less than mm which shows that the proposed method works very well for images without any strong artifact. Furthermore, the general performance of our proposed method for lumen segmentation is superior to other available methods in terms of the average Hausdorff distance, which is mm. Likewise, our method is able to segment the lumen when the image contains shadows ( mm) or side vessel ( mm) artifacts, which is by far the lowest distance to the gold standard.
If artifacts that are usually dark (low intensity value) regions, join the lumen and media regions, a type of leak will be created. This leak significantly increases the size and the mean intensity of the regions and generates irregular patterns that are very difficult to distinguish by the EREL extraction algorithm. Some of the inaccurate segmentations that are caused by the presence of the artifacts are shown in Figure 5. As can be seen in Figure 5, although the presence of the artifacts disrupts the detection of the media regions, most of the lumen regions can still be segmented accurately. This is also supported by the quantitative results presented in Table 3. Evidently, even in the presence of artifacts such as bifurcations, side vessels, and shadows, the lumen segmentation performance remains high, though the accuracy of the media segmentation drops. In fact, even in the worst case condition of lumen segmentation that happens when the image contains bifurcation artifacts, the average Hausdorff distance to the gold standard is only mm which is still lower than most of the other methods. On the other hand, the media segmentation is more sensitive to the presence of artifacts. For instance, the Hausdorff metric for media in some cases increases to mm for bifurcation and to mm for shadow artifact. Accordingly, it can be concluded that the lumen segmentation is robust to the common artifacts of the IVUS images.
The success of the proposed method depends strictly on the correct extraction of ERELs. In cases when continuous and clear boundaries do not exist between extremal regions, EREL cannot extract correct candidate regions for lumen and media. For example, in IVUS images taken with a 40 MHz catheter probe, some lumen pixels located on the boundaries adjacent to intima do not have clear connections to each other. A similar situation occurs for the boundaries between media and adventitia. This creates leakages that attach media region to the adventitia. So, the sequential process of EREL’s enumeration connects pixels from two semantically different regions and constructs a big connected component that includes both regions. Eventually, the leakage obstructs EREL’s region extraction capability for detecting regions. This leakage can also be created in IVUS 20 MHz images by various artifacts that somehow attach the media regions to the adventitia as illustrated in Figure 5.
Apart from the above, our proposed method offers several benefits. Not only is our method automatic, but works also solely on the B-mode images (in contrast to some methods available in the literature that work on the polar space unal2008shape ; taki2008automatic ; mendizabal2013segmentation ; zakeri2017automatic ) and, thus, it eliminates the need for transformation to the polar space. In addition, some methods sun2012parallel ; cardinal2006intravascular require the whole volume to be presented in order to segment the arterial walls. This makes it impossible to segment the lumen and media in online segmentation applications (during the in-vivo pullback procedures). On the other hand, our method delineates lumen and media contours using only the information available in the current frame and, hence, performs no redundant processing. Furthermore, due to the low computational complexity of the EREL faraji2015erel ; faraji2015extremal , every frame can be segmented in linear time based on the number of pixels in the frame. The average run time per frame in the proposed method over the test set of balocco2014standardized is seconds.
In this study, we investigated the suitability of a recently proposed region detector called Extremal Regions of Extremum Levels for detecting the lumen and media regions. The results of our experiments reveal that among the ERELs extracted from a single IVUS frame there exist regions with very low difference from the manually annotated data. Therefore, we presented a region selection approach to automatically segment the arterial walls from Intravascular Ultrasound images taken with a 20 MHz catheter probe. We embedded the mean intensity value, the entropy, and the boundary length of the regions into a single feature vector representing the texture information of the region. Then, our selection method looks for regions with higher stability score. The region that is most stable at the beginning section of the texture vector is considered as the lumen. Similarly, the last region with a stable textural variation is labeled as media.
We evaluated our segmentation method on the test set of a standard publicly available IVUS dataset using three common evaluation metrics; namely, JM, HD, and PAD. Extensive quantitative comparisons show the high accuracy of our proposed method. In general, the average HD for our lumen segmentation results is mm which is the closest distance to the gold standard among all existing methods. In addition, when no artifact was present, our method segments both lumen and media achieving average HD lower than mm and mm, respectively.
As reported in the best case results, among the extracted EREL regions, there exist regions very close to the actual structure of the arterial walls. In future work, we would like to devise a better selection strategy that chooses ERELs more accurately.
Since IVUS is completely radiation free, unlike X-ray and CT, in the future we would also like to investigate the feasibility of combining the segmentation results from individual slices to obtain 3D views of collateral arteries.
We gratefully acknowledge the assistance of Prof. Simone Balocco at the University of Barcelona for providing the labeling information on the existing artifacts in the dataset.
- (1) J. Frostegård, Sle, atherosclerosis and cardiovascular disease, Journal of internal medicine 257 (6) (2005) 485–495.
- (2) A. Katouzian, E. D. Angelini, S. G. Carlier, J. S. Suri, N. Navab, A. F. Laine, A state-of-the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images, IEEE Transactions on Information Technology in Biomedicine 16 (5) (2012) 823–834.
- (3) T. Ma, B. Zhou, T. K. Hsiai, K. K. Shung, A review of intravascular ultrasound-based multimodal intravascular imaging: the synergistic approach to characterizing vulnerable plaques, Ultrasonic imaging 38 (5) (2016) 314–331.
- (4) S. Balocco, C. Gatta, F. Ciompi, A. Wahle, P. Radeva, S. Carlier, G. Unal, E. Sanidas, J. Mauri, X. Carillo, et al., Standardized evaluation methodology and reference database for evaluating ivus image segmentation, Computerized Medical Imaging and Graphics 38 (2) (2014) 70–90.
- (5) G. Unal, S. Bucher, S. Carlier, G. Slabaugh, T. Fang, K. Tanaka, Shape-driven segmentation of the arterial wall in intravascular ultrasound images, IEEE Transactions on Information Technology in Biomedicine 12 (3) (2008) 335–347.
- (6) G. Mendizabal-Ruiz, I. A. Kakadiaris, A physics-based intravascular ultrasound image reconstruction method for lumen segmentation, Computers in biology and medicine 75 (2016) 19–29.
- (7) A. Taki, Z. Najafi, A. Roodaki, S. K. Setarehdan, R. A. Zoroofi, A. Konig, N. Navab, Automatic segmentation of calcified plaques and vessel borders in ivus images, International Journal of Computer Assisted Radiology and Surgery 3 (3-4) (2008) 347–354.
- (8) X. Zhu, P. Zhang, J. Shao, Y. Cheng, Y. Zhang, J. Bai, A snake-based method for segmentation of intravascular ultrasound images and its in vivo validation, Ultrasonics 51 (2) (2011) 181–189.
- (9) E. G. Mendizabal-Ruiz, M. Rivera, I. A. Kakadiaris, Segmentation of the luminal border in intravascular ultrasound b-mode images using a probabilistic approach, Medical image analysis 17 (6) (2013) 649–670.
- (10) F. S. Zakeri, S. K. Setarehdan, S. Norouzi, Automatic media-adventitia ivus image segmentation based on sparse representation framework and dynamic directional active contour model, Computers in Biology and Medicine.
- (11) S. Su, Z. Hu, Q. Lin, W. K. Hau, Z. Gao, H. Zhang, An artificial neural network method for lumen and media-adventitia border detection in IVUS, Computerized Medical Imaging and Graphics 57 (2017) 29–39.
- (12) J. Yan, D. Lv, Y. Cui, A novel segmentation approach for intravascular ultrasound images, Journal of Medical and Biological Engineering 37 (3) (2017) 386–394.
- (13) L. Lo Vercio, J. I. Orlando, M. del Fresno, I. Larrabide, Assessment of image features for vessel wall segmentation in intravascular ultrasound images, International journal of computer assisted radiology and surgery 11 (8) (2016) 1397–1407.
- (14) R. M. Haralick, Statistical and structural approaches to texture, Proceedings of the IEEE 67 (5) (1979) 786–804.
- (15) M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. B. Moeslund, Erel: Extremal regions of extremum levels, in: Image Processing (ICIP), 2015 IEEE International Conference on, IEEE, 2015, pp. 681–685.
- (16) M. Faraji, J. Shanbehzadeh, K. Nasrollahi, T. Moeslund, Extremal regions detection guided by maxima of gradient magnitude, Image Processing, IEEE Transactions on.
- (17) F. Destrempes, M.-H. R. Cardinal, L. Allard, J.-C. Tardif, G. Cloutier, Segmentation method of intravascular ultrasound images of human coronary arteries, Computerized Medical Imaging and Graphics 38 (2) (2014) 91–103.
- (18) C. P. Loizou, C. S. Pattichis, Despeckle filtering algorithms and software for ultrasound imaging, Synthesis lectures on algorithms and software in engineering 1 (1) (2008) 1–166.
- (19) R. Sedgewick, K. Wayne, Algorithms, 4th Edition., Addison-Wesley, 2011.
- (20) B. Iglewicz, D. C. Hoaglin, How to detect and handle outliers, Vol. 16, Asq Press, 1993.
L. Shapiro, G. C. Stockman, Computer vision. 2001, ed: Prentice Hall.
- (22) R. Szeliski, Computer vision: algorithms and applications, Springer Science & Business Media, 2010.
- (23) R. Downe, A. Wahle, T. Kovarnik, H. Skalicka, J. Lopez, J. Horak, M. Sonka, Segmentation of intravascular ultrasound images using graph search and a novel cost function, in: Proc. 2nd MICCAI workshop on computer vision for intravascular and intracardiac imaging, Citeseer, 2008, pp. 71–79.
- (24) F. Ciompi, O. Pujol, C. Gatta, M. Alberti, S. Balocco, X. Carrillo, J. Mauri-Ferre, P. Radeva, Holimab: A holistic approach for media–adventitia border detection in intravascular ultrasound, Medical image analysis 16 (6) (2012) 1085–1100.
- (25) F. Molinari, K. M. Meiburger, G. Zeng, A. Nicolaides, J. S. Suri, Caudles-ef: carotid automated ultrasound double line extraction system using edge flow, Journal of digital imaging 24 (6) (2011) 1059–1077.
- (26) Z. Sun, C. Liu, A parallel method for segmenting intravascular ultrasound image sequence, in: Mechanical and Electronics Engineering III, Vol. 130 of Applied Mechanics and Materials, Trans Tech Publications, 2012, pp. 2051–2055. doi:10.4028/www.scientific.net/AMM.130-134.2051.
- (27) M.-H. Cardinal, J. Meunier, G. Soulez, R. L. Maurice, É. Therasse, G. Cloutier, Intravascular ultrasound image segmentation: a three-dimensional fast-marching method based on gray level distributions, IEEE transactions on medical imaging 25 (5) (2006) 590–601.