IVUS Image Segmentation Using Superpixel-Wise Fuzzy Clustering and Level Set Evolution

Abstract

Reliable detection of the media-adventitia border (MAB) and the lumen-intima border (LIB) in intravascular ultrasound (IVUS) images remains a challenging task that is of high clinical interest. In this paper, we propose a superpixel-wise fuzzy clustering technique modified by edges, followed by level set evolution (SFCME-LSE), for automatic border extraction in 40 MHz IVUS images. The contributions are three-fold. First, the usage of superpixels suppresses the influence of speckle noise in ultrasound images on the clustering results. Second, we propose a region of interest (ROI) assignment scheme to prevent the segmentation from being distracted by pathological structures and artifacts. Finally, the contour is converged towards the target boundary through LSE with an appropriately improved edge indicator. Quantitative evaluations on two IVUS datasets by the Jaccard measure (JM), the percentage of area difference (PAD), and the Hausdorff distance (HD) demonstrate the effectiveness of the proposed SFCME-LSE method. SFCME-LSE achieves the minimal HD of 1.20 ± 0.66 mm and 1.18 ± 0.70 mm for the MAB and LIB, respectively, among several state-of-the-art methods on a publicly available dataset.

1. Introduction

Atherosclerosis involves pathological structures (stenosis, vulnerable plaque, etc.) within the arterial wall, leading to a significant reduction in blood flow. Atherosclerotic plaque can become fragile, potentially leading to cardiovascular diseases, such as angina, myocardial infarction, stroke, and sudden cardiac death [1]. Intravascular ultrasound (IVUS) is an in-vivo medical imaging technique capable of providing tomographic grayscale images of the vessel in real-time [2]. IVUS is commonly used in the diagnosis of atherosclerosis. Two types of borders in IVUS images, the lumen-intima border (LIB) and the media-adventitia border (MAB), are of substantial clinical interest. Automatic identification of these two borders is crucial for assessing the plaque burden and mechanical properties of the coronary artery [2].

Reliable border detection is obstructed by complicated intravascular structures, various forms of impediments, and intrinsic artifacts. Many methodologies have been proposed for IVUS image segmentation. We briefly summarize some of the published algorithms next.

Pixel classification followed by deformable models is the most common strategy for IVUS image segmentation. The classification took the morphological, textural, and grayscale features into consideration and was achieved by the extreme learning machine [3], dictionary learning in the sparse representation framework [4], support vector machines [5,6,7], or artificial neural networks [8]. An assessment of the capability of different features for segmenting IVUS images was given by Lo Vercio et al. [7]. In other deformable model-based methods, a global intensity term and an oriented smooth gradient were developed for the initial border extraction [9]. The detected border of the previous frame was utilized as an initial estimation of the border of the current frame in an IVUS sequence [10]. A 3D helical contour initialized as a centered helix model with a fixed radius was deformed to complete the segmentation [11,12]. Among these methods, feature-based classification automatically gives a relatively accurate initial border, but the extraction of textural features and the training of classifiers are time consuming.

The fast marching method achieves segmentation by propagating an interface to the boundary under a speed function. Fast marching methods for IVUS sequences generally model the gray-level distribution of vessel anatomical structures as a mixture of probability density functions (PDFs). Rayleigh PDFs [13,14], gamma PDFs [15], and the intensity gradient [14,15,16] were introduced to construct the velocity function of the propagating interface. However, these algorithms assume that regions in the image have a statistically distinctive distribution, which is not always accurate, as IVUS images contain various forms of diseased structures. Among other categories of methods, the graph-based approach [17,18], the Grow-Cut algorithm [19], the nonparametric statistical approach [20], and the deep learning architecture [21,22] were introduced to IVUS image segmentation. The main limitation of deep learning for IVUS image segmentation lies in the lack of enough labeled data, which must be provided by IVUS experts.

Despite the considerable attempts that have been devoted to IVUS image segmentation, no approved universal method exists that guarantees a successful segmentation. In this paper, we propose a three-fold automatic border detection strategy. First, we conduct superpixel-wise fuzzy clustering modified by edges (SFCME). Then, the clustering maps are fed into a framework that assigns regions of interest (ROIs) to different vascular tissues. The initial border is extracted according to the assignment and finally converged by edge-based level set evolution (LSE). The proposed SFCME-LSE method has a simple and fast implementation without the need for labeled training data. It takes advantage of the useful information in images while reducing interferences. Specifically, the superpixel-wise clustering method is based on the regional gray information, suppressing the influence of speckle noise in the ultrasound images on the results. ROIs are assigned according to the anatomical and morphological descriptors in the IVUS images. The deformation of contours in LSE mainly relies on edge information.

The rest of the paper proceeds as follows. In Section 2, we present the motivation and correlated theory of our method. Section 3 details the algorithm. The effectiveness of the proposed method is validated by statistical results in Section 4. Section 5 gives the extended discussion, and Section 6 concludes the paper.

2. Motivation

As multiple pathological structures and artifacts exist in IVUS images, it is difficult for a single segmentation method to detect the MAB and LIB accurately. We use a separation/aggregation strategy to prevent the extraction of borders from being distracted by those impediments. A clustering technique followed by ROI assignment roughly identifies important vascular components, contributing to the localization of the initial boundary.

2.1. Separation of Vascular Components

Clustering methods have been widely used in image segmentation. Employing a fuzzy c-means (FCM) technique can determine approximate ROIs in medical images. The algorithm is related to the minimization of an objective function [23]:

$$J={{\displaystyle \sum }}_{i=1}^{N_i}{{\displaystyle \sum }}_{c=1}^{N_c}{u}_{ci}{}^m\Vert I\left(i\right)-v_c{\Vert }^2, \mathrm{subject}\mathrm{to}{{\displaystyle \sum }}_{c=1}^{N_c}{u}_{ci}=1, 0\le u_{ci}\le 1,$$

(1)

where $i$ and $c$ are the pixel index and the cluster index, respectively. $N_i$ is the number of pixels in the image. $N_c$ denotes the number of clusters. $u_{ci}$ indicates the membership between the ith pixel and the cth cluster, and $v_c$ is the centroid of the cth cluster. $I$ represents the attribute (intensity, color, etc.) of an image, and in this paper, $I\left(i\right)$ is set to the gray value of pixel $i$. $m$ is the weighting exponent that controls the fuzziness of the clustering and is set to 2 in most segmentation tasks.

$J$ is optimized to its local minimum by updating $v_c$ and $u_{ci}$ iteratively [23]:

$$v_c=\frac{{{\displaystyle \sum }}_{i=1}^{N_i}{u}_{ci}{}^{m}I\left(i\right)}{{{\displaystyle \sum }}_{i=1}^{N_i}{u}_{ci}{}^m}$$

(2)

$$u_{ci}=\frac{\Vert I\left(i\right)-v_c{\Vert }^{-2/\left(m-1\right)}}{{{\displaystyle \sum }}_{k=1}^{N_c}\Vert I\left(i\right)-v_k{\Vert }^{-2/\left(m-1\right)}}$$

(3)

Many methods based on the FCM provide ways to obtain better segmentation results by absorbing the spatial information. The intensities of pixels in a window around a pixel $i$ were used as the spatial constraint for $i$ [24,25]. The spatial information was incorporated into the fuzzy membership function $u_{ci}$ [26]. A fuzzy factor was proposed to flexibly change the influence of pixels within the local window around pixel $i$, according to their Euclidean distance from $i$ [27]. The image segmentation effect was improved by incorporating the adaptive local spatial information into the objective function [28]. Through morphological gradient reconstruction, superpixels were obtained to provide adaptive and irregular local spatial neighborhoods for color image segmentation [29].

Simple linear iterative clustering (SLIC) can efficiently generate superpixels [30]. In this paper, we integrate SLIC into the clustering process to provide adaptive spatial information and reduce the amount of processing elements in the fuzzy clustering procedure.

2.2. Assignment of Vascular Components

Despite the extreme difference in the appearance of IVUS images, a coronary cross-section is generally composed of a lumen and a coronary vessel wall composed of three layers: The intima, the media, and the adventitia. Additionally, several types of structures (calcification, soft plaque) and artifacts (ring-down artifacts, guidewire artifacts, shadow artifacts) are included in IVUS images. In this paper, calcification refers to the calcified plaque, and soft plaque includes fibrotic plaque. An example image with different components labeled is shown in Figure 1.

The adventitia, plaque, and calcification regions are highly echo-reflective and appear as bright tissues in the image, while the lumen appears darker. Calcifications usually produce signal attenuation behind them, resulting in shadow artifacts. The intima and the media are very thin and not always visual in IVUS images. Ring-down artifacts are characterized by a bright halo around the black catheter area in the center of the image.

In this paper, we consider that the catheter zone $R_c$ consists of pixels whose distance from the image center $i_c$ is less than the catheter radius $D_c$:

$$R_c=\bigcup i, d\left(i,i_c\right)\le D_c,$$

(4)

where $d\left(i,i_c\right)$ is the Euclidean distance between pixels $i$ and $i_c$. The background zone $R_b$ is the region outside the inscribed circle of the image:

$$R_b=\bigcup i, d\left(i,i_c\right)\ge D_b,$$

(5)

where $D_b$ is the radius of the inscribed circle.

Inspired by the grayscale differences in these image components, as well as their anatomical and spatial features, we propose a ROI assignment algorithm, which is detailed in Section 3.3.

2.3. Convergence of Contours

The level set method remains a widely used technique for aligning the initial contour to the target boundary. It models a closed curve $S$ as the zero level set of a Lipschitz function $\varphi$. The convergence of $S$ involves minimizing an energy function $E\left(\varphi \right)$ with respect to the level set function [31]:

$$S\left(t\right)=\left\{\left(x,y,t\right)|\varphi \left(x,y,t\right)=0\right\},$$

(6)

$$\frac{\partial \varphi }{\partial t}=-\frac{\partial E\left(\varphi \right)}{\partial \varphi },$$

(7)

where $t$ represents the evolution time, and $\left(x,y\right)$ are the Cartesian coordinates of a pixel in the image.

Existing LSE models can be generally grouped into region-based and edge-based models. The distance regularized LSE (DRLSE) method slows down the shrinkage or expansion of $S$ at image edges [32]. The region-based Chan–Vese model deforms $S$ according to the gray difference between regions inside and outside the contour [33]. A kernel function was integrated into the Chan–Vese model for better effectiveness in inhomogeneous images [34]. A combined data fidelity term was integrated into the energy function for efficiently handling images with additive and multiplicative noise [35].

Level set techniques often require initialization and do not guarantee a global minimum. After obtaining the initial contour from superpixel-based clustering results, we choose edge-based LSE with a new edge indicator to refine the contour.

3. Method

The proposed method is three-fold, and its workflow is depicted in Figure 2. Each frame is pre-processed, and clustering is executed first to separate vascular components of interest into different ROIs. Then, the obtained ROIs are assigned into five vessel tissues through a framework controlled by the thresholds of three indicators. At the same time, the border is updated along with the ROI assignment. Finally, edge-based LSE drives the initial border towards the target.

3.1. Pre-Processing

Image intensities are normalized to the interval $\left[0, 1\right]$. Calibration squares with an intensity equal to 1 are replaced by the average intensity of 24 connected pixel neighbors. The catheter zone $R_c$ is blackened to remove ring-down artifacts. The image after those treatments is denoted by $I_o$. Then, contrast-limited adaptive histogram equalization [36] is applied to $I_o$, generating the edge-sharpened image $I$.

3.2. Superpixel-Wise Fuzzy Clustering Modified by Edges

We adopt SLIC [30] to obtain preliminary over-segmentation that provides the FCM technique with adaptive and irregular local spatial neighborhoods. Moreover, superpixel-wise clustering is able to achieve low computational complexity by reducing the number of processing elements. With the number of iterations, the desired number of superpixels, and compactness set to 20, 333, and 13, respectively, with SLIC, we can obtain an accurate over-segmentation for post-processing. Finally, the catheter zone $R_c$ and the background zone $R_b$ are both considered superpixels.

The obtained local spatial information is incorporated into the objective function of the FCM algorithm, defined by:

$$J={{\displaystyle \sum }}_{s=1}^{N_s}{{\displaystyle \sum }}_{c=1}^{N_c}{P}_s{u}_{cs}{}^m\Vert q_s-v_c{\Vert }^2, \mathrm{subject}\mathrm{to}{{\displaystyle \sum }}_{c=1}^{N_c}{u}_{cs}=1, 0\le u_{cs}\le 1,$$

(8)

where $s$ is the superpixel index and $N_s$ is the number of superpixels. $P_s$ is the number of pixels in the sth superpixel $R_s$. The mean intensity $q_s$ in $R_s$ is calculated as:

$$q_s=\frac{1}{P_s}{{\displaystyle \sum }}_{i\in R_s}I\left(i\right)$$

(9)

Similar to Equations (2) and (3), the membership function $u_{cs}$ and cluster center $v_c$ are updated according to:

$$v_c=\frac{{{\displaystyle \sum }}_{s=1}^{N_s}{u}_{cs}{}^m{P}_s{q}_s}{{{\displaystyle \sum }}_{s=1}^{N_s}{u}_{cs}{}^m{P}_s}$$

(10)

$$u_{cs}=\frac{\Vert q_s-v_c{\Vert }^{-2/\left(m-1\right)}}{{{\displaystyle \sum }}_{k=1}^{N_c}\Vert q_s-v_k{\Vert }^{-2/\left(m-1\right)}}$$

(11)

Each superpixel belongs to the cluster that has the most of membership belonging. $N_c$ takes the value of 3, and three clustering maps are obtained. The cluster denoted by $B_{1o}$, which has the smallest white areas, consists of ROIs located inside the adventitia, calcification, and soft plaque areas. The cluster denoted by $B_{2o}$ whose center is white contains ROIs belonging to shadow artifacts and soft plaque, as shown in Figure 2.

After clustering, ROIs belonging to different tissues may still have weak connections in $B_{1o}$. To fully cut these connections, $B_{1o}$ minus an edge map $B_e$ and is processed morphologically. The resulting map is denoted by $B_1$.

$$B_1=\left(\left(B_{1o}-B_e\right)\ominus b_{disc}\right)\oplus b_{disc},$$

(12)

where $b_{disc}$ is a disc-shaped structuring element. $\ominus$ and $\oplus$ denote morphological erosion and the dilation operation, respectively.

Edges are actually high-frequency areas in an image. If lower-frequency information is rejected by some sort of filter, higher frequencies will remain, which are edges [37]. $B_e$ is computed according to the maximum magnitude response of a bank of 2D Morlet wavelet kernels with dimensions equal to $1/32$ the size of the image, as shown in Figure 3.

3.3. ROI Assignment Algorithm

As detailed in Section 2.2, tissues in IVUS images usually have an obvious relative position distribution, and they have roughly different gray levels. Inspired by these observations, we propose an automatic algorithm that assigns ROIs in $B_1$ and $B_{2o}$ to five categories: Calcification, adventitia, soft plaque, lumen, and shadow. Additionally, an iterative boundary updating method works with the ROI assignment algorithm. The two processes regulate and promote each other. Specifically, ROIs are processed from largest to smallest by area. The ROI being processed is denoted by $R_{on}$.

We define a grayscale indicator $D_g$, an anatomical indicator $D_a$, and a spatial indicator $D_s$ to control the assignment. $D_g$ is the mean intensity of pixels in $R_{on}$ divided by the mean intensity of pixels in all ROIs in $B_1$:

$$D_g=\frac{\frac{1}{P_{on}}{{\displaystyle \sum }}_{i\in R_{on}}{I}_o\left(i\right)}{\frac{1}{P_B}{{\displaystyle \sum }}_{i\in B_1}{I}_o\left(i\right)},$$

(13)

where $P_{on}$ and $P_B$ are the number of pixels in $R_{on}$ and $B_1$, respectively. Generally, the order from largest to smallest in terms of $D_g$ is calcification, adventitia, soft plaque, lumen, and shadow.

Calcific regions usually produce signal attenuation behind them, appearing in the image as a shadow outside the calcification. The ratio of the mean intensity of an ROI to the mean intensity of regions outside the ROI is presented to detect calcifications [38]. However, some calcifications may leave a small shadow, and not all areas outside bright calcifications belong to the shadow. In this paper, we assume a calcification leaves a shadow with a thickness $L_{out}$ outside it. $D_a$ is defined as:

$$D_a=100·\frac{1}{D_g}·{\left(\frac{\frac{1}{P_{out}}{{\displaystyle \sum }}_{i\in R_{out}}{I}_o\left(i\right)}{\frac{1}{P_{on}}{{\displaystyle \sum }}_{i\in R_{on}}{I}_o\left(i\right)}\right)}^2,$$

(14)

where $R_{out}$ is the region outside $R_{on}$ with a thickness $L_{out}$, as depicted in Figure 4. $P_{out}$ is the number of pixels in $R_{out}$. Usually, the order from largest to smallest in terms of $D_a$ is lumen, soft plaque, adventitia, and calcification.

$D_s$ quantifies the proximity of $R_{on}$ to the border, defined as the minimum distance between $R_{on}$ and the current MAB.

$$D_s\left(R_{on},MAB\right)=\pm \underset{i_a\in R_{on}}{\min }\left\{\underset{i_b\in MAB}{\min }\left\{d\left(i_a,i_b\right)\right\}\right\},$$

(15)

where $i_a$ and $i_b$ are two points in $R_{on}$ and the MAB, respectively. $D_s\left(R_{on},MAB\right)$ is positive if $R_{on}$ is outside the MAB, and vice versa.

The workflow of the assignment algorithm is illustrated in

Table 1. The workflow of the region of interest (ROI) assignment algorithm.

Step (1)	MAB = rectangular bounding border of ROIs in $B_1$; Vessel center = the image center (Figure 5a); $R_{on}$ = the largest ROI in $B_1$; $B_1\left(R_{on}\right)=0$; Calcification recognition $\stackrel{\mathrm{else}}{\to }$ $R_{on}$ belongs to the adventitia; MAB update (Figure 5b); If ${\theta }_{adven}>T_{\theta }$, go to Step (3); else, go to Step (2).
Step (2)	$R_{on}$ = the largest ROI in $B_1$; $B_1\left(R_{on}\right)=0$; Processing for guidewire artifact; Execute calcification, adventitia, soft plaque, and lumen recognition sequentially; MAB update (Figure 5c).
Step (3)	A circle is fitted to the MAB; Vessel center = the center of the fitted circle (Figure 5c); MAB update (Figure 5d); If ${\theta }_{adven}>T_{\theta }$, go to Step (5); else, go to Step (4).
Step (4)	$R_{on}$ = the largest ROI in $B_1$; $B_1\left(R_{on}\right)=0$; $A_{on}=\mathrm{Area}\left(R_{on}\right)$; Processing for guidewire artifact; If $A_{on}<T_{area}$, go to Step (7); Else, execute adventitia, soft plaque, and lumen recognition sequentially; MAB update (Figure 5e–g); If ${\theta }_{adven}>T_{\theta }$, go to Step (5); else, go to Step (4).
Step (5)	ROIs outside MAB are deleted in $B_1$.
Step (6)	$R_{on}$ = the largest ROI in $B_1$; $B_1\left(R_{on}\right)=0$; $A_{on}=\mathrm{Area}\left(R_{on}\right)$; Processing for guidewire artifact; If $A_{on}<T_{area}$, go to Step (7); Else, execute soft plaque and lumen recognition sequentially; Go to Step (6).
Step (7)	ROIs outside MAB and the ROI containing $R_c$ are deleted in $B_{2o}$, the result is denoted by $B_2$ (Figure 5h).
Step (8)	$R_{on}$ = the largest ROI in $B_2$; $B_2\left(R_{on}\right)=0$; $A_{on}=\mathrm{Area}\left(R_{on}\right)$; If $A_{on}<T_{area}$, go to Step (9); Else, execute shadow and soft plaque recognition sequentially (Figure 5i); Go to Step (8).
Step (9)	LIB extraction (Figure 5j).

with a graphical example in Figure 5. Additional explanations are described below. The parameters involved in the method are listed in

Table 2. Parameter selection in the proposed method.

Symbol	Definition	Reference	Value
			Dataset I	Dataset II
$D_c$	The radius of the catheter zone $R_c$	Equation (4)	43 pixels
$L_{out}$	The thickness of the shadow left by a calcification	Figure 4	23 pixels
$T_d$	The distance threshold for guidewire artifact processing	Section 3.3.1	3 pixels
$T_{\theta }$	The angle threshold	Section 3.3.2	281°
$T_{area}$	The area threshold	Section 3.3.2	$100{\mathrm{pixels}}^2$
$T_{a1}$	Thresholds for the anatomical indicator $D_a$	Section 3.3.3	12.4	18.8
$T_{a2}$			9.4	12.4
$T_{a3}$			11.5	17.8
$T_{a4}$			38
$T_s$	The threshold for the spatial indicator $D_s$		−20
$T_{g1}$	Thresholds for the grayscale indicator $D_g$		0.73	0.81
$T_{g2}$			0.61	0.78
$T_{g3}$			0.55
$T_{g4}$			0.16
$\mu$	Parameters controlling level set evolution	Equation (17)	0.2
$\beta$			1
$\nu$			−0.03 (MAB)/0.03 (LIB)

.

3.3.1. Processing for Guidewire Artifacts

If the minimal distance between $R_{on}$ and the catheter region $R_c$ is smaller than a specific threshold $T_d$, we consider that $R_{on}$ contains guidewire artifacts. Pixels touching $R_c$ as well as their radial outside pixels belong to the guidewire artifact and are excluded from $R_{on}$ before the assignment, as documented in Figure 5e–g.

3.3.2. MAB Update

IVUS images can be viewed as 360-degree cross-sections of the vessel wall. Starting from the vessel center, the first adventitia pixel in the radial direction at each angle is sought. If no adventitia pixel is found, the $L_{out}$ pixel outside the last calcification pixel is selected. The MAB is then obtained according to the radial linear interpolation of the selected pixels, as shown in Figure 5g. Finally, topological constraints are imposed, i.e., the MAB is outside the soft plaque, calcification, and lumen.

The update of the MAB runs once an ROI is assigned, and is terminated when the angle ${\theta }_{adven}$ of the arc composed of adventitia pixels exceeds a threshold $T_{\theta }$ or all ROIs larger than the threshold $T_{area}$ have been assigned.

3.3.3. Tissue Recognition

ROIs belonging to different tissues are scattered in the metric space formed by three indicators. The graphical description will be given in Section 5. We use five judgments of the priority (from high to low) to complete the recognition: ① if ($D_s<0$ and $D_a<T_{a1}$) or ($D_a<T_{a2}$), $R_{on}$ belongs to a calcification; ② if ($D_s>0$) or ($D_s=0$ and $T_{a3}\le D_a<T_{a4}$), $R_{on}$ belongs to the adventitia; ③ if ($D_s\le 0$ and $D_g>T_{g1}$) or ($T_s<D_s\le 0$ and $D_g>T_{g2}$), $R_{on}$ belongs to soft plaque; ④ if ($T_s<D_s\le 0$ and $D_g>T_{g3}$), $R_{on}$ belongs to lumen; ⑤ if ($D_g<T_{g4}$), $R_{on}$ belongs to a shadow.

3.3.4. LIB Extraction

After all the ROIs have been assigned, radial searching followed by linear interpolation is employed to locate the LIB inside the adventitia, calcification, soft plaque, and shadow, and outside the lumen, as shown in Figure 5j.

3.4. Level Set Evolution

As the above operations are generally based on regional information and the initial border is close to the target, we adopt edge-based DRLSE for further contour convergence, formulated as [32]:

$$E\left(\varphi \right)=\mu \left\{\frac{1}{2}{{\displaystyle \int }}_{\mathsf{\Omega }}^{}{\left(\left|\nabla \varphi \right|-1\right)}^{2}dxdy\right\}+\beta \left\{{{\displaystyle \int }}_{\mathsf{\Omega }}^{}g\delta \left(\varphi \right)\left|\nabla \varphi \right|dxdy\right\}+\nu \left\{{{\displaystyle \int }}_{\mathsf{\Omega }}^{}gH\left(\varphi \right)dxdy\right\}$$

(16)

The energy function is minimized through:

$$\frac{\partial \varphi }{\partial t}=\mu ·\left\{{\nabla }^2\varphi -div\left(\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)\right\}+\beta ·\left\{\delta \left(\varphi \right)div\left(g\frac{\nabla \varphi }{\left|\nabla \varphi \right|}\right)\right\}-\nu ·g\delta \left(\varphi \right),$$

(17)

where the first term is a distance regularization term maintaining the regularity of the level set function $\varphi$, the second term and the third term are for minimizing the weighted length of the contour $S$ and the weighted area inside $S$, respectively. The factors $\mu$, $\beta$, and $\nu$ are weights regulating the contribution of each term. Ω is the image domain. $\nabla$, ${\nabla }^2$, and $div$ denote the gradient, Laplacian, and divergence operators, respectively. $\delta \left(\varphi \right)$ is the Dirac delta function, which is the derivative of the Heaviside function $H\left(\varphi \right)$ [32]. $g$ is the edge indicator that slows down the shrinkage or expansion of $S$ when it arrives at edges [32].

$$g=\frac{1}{1+{\left|\nabla G_{\sigma }\ast I\right|}^2}$$

(18)

Edges are not clear in ultrasound images. Due to noise, a large image gradient may appear at pixels not located on desired boundaries, interfering with the deformation of the contour. We propose a new edge indicator that highlights the main edge information for better contour convergence.

$$g_{new}=\max \left\{\overline{g},\overline{1-B_e}\right\},$$

(19)

where $\overline{·}$ denotes unit normalization.

4. Experiments and Results

4.1. Materials and Evaluation Measures

The experimental evaluation of the proposed SFCME-LSE method is carried out on two IVUS datasets. Dataset I contains 326 frames from eight pullbacks acquired from four patients at the Department of Cardiology, Zhongshan Hospital of Fudan University, Shanghai, China. These images do not disclose patient information, and informed consent to use these images has been obtained from the patients and the hospital. One set of manual labeling is provided as the ground truth. The imaging system used for the acquisition is an iLab IVUS (Boston Scientific, Fremont, United States) equipped with a 40 MHz OptiCross catheter. Dataset II, which is publicly available, consists of 77 frames from 22 patients [39]. The imaging system for Dataset II is also an iLab IVUS, but with a different catheter, the 40 MHz Atlantis SR 40 Pro catheter. Dataset II has three sets of annotations from two clinical experts for intra- and inter-observer evaluations. Dataset I and Dataset II present a good diversity of artifacts, and some frames include multiple artifacts. Specifically, the number of frames containing plaque (or calcification), bifurcation, stents, side vessels, shadow artifacts, and guidewire artifacts are 69, 16, 22, 8, 42, and 36, respectively, in Dataset II.

The algorithm is implemented in MATLAB R2018a on a Dell Precision T7610 with Intel(R) Xeon(R) E5-2637 v2 3.5 GHz CPU.

Three performance measures are usually chosen for the quantitative analysis in IVUS segmentation [39]. The Jaccard measure (JM) weighs the similarity between the segmented area $R_{seg}$ and the manually defined area $R_{truth}$ and is expressed as:

$$JM=\frac{\left|R_{seg}\bigcap R_{truth}\right|}{\left|R_{seg}\bigcup R_{truth}\right|}$$

(20)

The percentage of area difference (PAD) which computes the area difference between $R_{seg}$ and $R_{truth}$ is defined as:

$$PAD=\frac{\left|Area\left(R_{seg}\right)-Area\left(R_{truth}\right)\right|}{\left|Area\left(R_{truth}\right)\right|},$$

(21)

where $Area\left(·\right)$ represents the area calculation. The Hausdorff distance (HD) quantifies the distance between the resulting contour $S_{seg}$ and the manual annotation $S_{truth}$:

$$HD=\underset{i_a\in S_{seg}}{\max }\left\{\underset{i_b\in S_{truth}}{\min }\left\{d\left(i_a,i_b\right)\right\}\right\},$$

(22)

where $i_a$ and $i_b$ are two points in $S_{seg}$ and $S_{truth}$, respectively. The higher the JM is, the smaller the PAD and HD, the better the performance of the segmentation method.

4.2. Self-Evaluation

Self-evaluation of the proposed method is carried out on Dataset I. The results of experiment 1 (the proposed method without LSE), experiment 2 (the proposed method with LSE based on the edge indicator $g$), and experiment 3 (SFCME-LSE) are compared to understand the contributions of the individual components of the SFCME-LSE to the overall performance. Further, we add Gaussian noise to images to evaluate the anti-noise performance of the SFCME-LSE. Images from Dataset I already contain a high amount of noise. However, these original images are viewed noise-free and contaminated by additional noise with a specified signal-to-noise ratio (SNR). The real SNR of noisy images is lower than the specified value, as a result. The means and standard deviations of the numerical metrics for the MAB and LIB are summarized in

Table 3. The performance of the proposed method on Dataset I. The format is mean (standard deviation).

		Experiment 1	Experiment 2	Experiment 3	SFCME-LSE on Images with Different SNR
					70 dB	50 dB	30 dB	27 dB	25 dB	23 dB	20 dB
MAB	JM	0.87 (0.06)	0.89 (0.07)	0.91 (0.07)	0.87 (0.13)	0.87 (0.12)	0.87 (0.12)	0.87 (0.12)	0.84 (0.16)	0.79 (0.25)	0.66 (0.33)
	HD	0.90 (0.49)	0.65 (0.44)	0.65 (0.43)	0.89 (0.80)	0.88 (0.79)	0.87 (0.81)	0.89 (0.81)	1.02 (1.05)	1.17 (1.17)	0.62 (0.37)
	PAD	0.10 (0.09)	0.09 (0.10)	0.07 (0.09)	0.11 (0.17)	0.10 (0.14)	0.12 (0.21)	0.14 (0.26)	1.19 (1.41)	1.20 (1.29)	0.33 (0.35)
LIB	JM	0.73 (0.10)	0.80 (0.09)	0.81 (0.09)	0.80 (0.10)	0.80 (0.09)	0.80 (0.10)	0.78 (0.12)	0.77 (0.13)	0.75 (0.15)	0.67 (0.21)
	HD	1.41 (0.59)	0.88 (0.57)	0.86 (0.57)	0.95 (0.61)	0.97 (0.60)	0.96 (0.58)	1.06 (0.64)	1.07 (0.68)	1.11 (1.65)	1.33 (0.76)
	PAD	0.32 (0.23)	0.15 (0.15)	0.14 (0.13)	0.15 (0.13)	0.16 (0.14)	0.18 (0.19)	0.25 (0.47)	1.27 (1.52)	1.25 (1.23)	0.33 (0.26)

.

From a clinical perspective, the quality of a method might be analyzed intuitively according to the percentage of frames lying below a progressive threshold of the PAD, as shown in Figure 6a.

The results obtained in experiment 1 demonstrate the effectiveness of the proposed border initialization technology. There are substantial improvements in the HD from experiment 1 to experiment 3, indicating the convergence of edge-based LSE on the contour. We achieved a modest improvement in the JM from 0.89 using the edge indicator $g$ to 0.91 using $g_{new}$ in detecting the MAB, and from 0.80 to 0.81 in detecting the LIB. Additionally, high JMs, low HDs, and small PADs in experiment 3 illustrate to some extent the ability of SFCME-LSE for IVUS image segmentation. Further, the performance of SFCME-LSE is acceptable on images with SNR higher than 30 dB, as illustrated in Table 3 and Figure 6b.

In Figure 7, we present Bland–Altman plots and linear regression plots for $Area\left(R_{seg}\right)$ and $Area\left(R_{truth}\right)$, visualizing the robustness of the algorithm to the size of the vessel cross-sections. Specifically, the linear regression analysis indicates correlation with $R^2=0.95$ ($p=1.24\times {10}^{-214}$) and $R^2=0.85$ ($p=1.99\times {10}^{-135}$) for MAB and LIB detection, respectively. Bland–Altman plots indicate a slight bias of 1.33 ${\mathrm{mm}}^2$ and 0.60 ${\mathrm{mm}}^2$ for MAB and LIB detection, respectively.

4.3. Comparative Evaluation

Comparative evaluation with other automatic methods is carried out on Dataset II, and the results are listed in

Table 4. Comparison of the proposed SFCME-LSE method with other published automatic methods. Empty cells mean that the researchers have not given the corresponding result.

	MAB			LIB			Time(s)/Frame	Hardware Used
	JM	HD	PAD	JM	HD	PAD
SFCME-LSE	0.83 (0.10)	1.20 (0.66)	0.12 (0.11)	0.78 (0.10)	1.18 (0.70)	0.16 (0.15)	11.09	Xeon 3.5 GHz
Kermani et al. [20]	0.75 (0.13)	1.32 (0.99)	0.12 (0.12)	0.77 (0.13)	1.46 (1.23)	0.16 (0.15)	---	---
Wang et al. [3]	0.83 (0.09)	1.27 (0.67)	0.12 (0.13)	---	---	---	272.92	Core 4, 2.67 GHz
Essa et al. [18]	0.84 (0.10)	1.22 (0.72)	0.13 (0.15)	---	---	---	---	---
Balocco et al. [19]	---	---	---	0.72 (0.12)	1.70 (1.09)	0.22 (0.14)	13	Core 2 Duo, 2.13 GHz
Intra-observer	0.91 (0.07)	0.85 (0.60)	0.06 (0.07)	0.92 (0.06)	0.67 (0.52)	0.05 (0.06)
Inter-observer	0.87 (0.11)	1.14 (1.00)	0.11 (0.14)	0.86 (0.10)	1.04 (0.95)	0.10 (0.10)

. SFCME-LSE outperforms existing methods in terms of the HD, with 1.20 ± 0.66 mm and 1.18 ± 0.70 mm for MAB and LIB detection, respectively. Additionally, our method has a competitive result over those published methods in terms of the JM and PAD. Moreover, SFCME-LSE has an important advantage of a simple implementation without the time-consuming extraction of complex texture features.

Finally, a qualitative assessment is performed by comparing the segmentation and the ground truth on several exemplary frames extracted from Dataset I and Dataset II, as displayed in Figure 8.

5. Discussion

In this paper, we propose the SFCME-LSE method for the extraction of the MAB and LIB in IVUS images. Next, we analyze the method in detail from four aspects: The benefit of superpixel-wise fuzzy clustering for extracting ROIs, the reliability of the ROI assignment framework, the parameter selection, and the assessment of the strengths and weaknesses of the method.

In Figure 9, we present clustering results of SFCME and of FCM with spatial information [26]. Due to the intensity inhomogeneity and large amounts of speckle noise invariably existing in ultrasound images, the results of pixel-wise fuzzy clustering appear discontinuous. The integration of SLIC superpixels improves the ability of clustering to group pixels from the same tissue into one ROI, which simplifies the assignment of ROIs. Moreover, SFCME takes approximately 0.69 s per frame in our environment, while the other method takes 2.32 s.

Among different IVUS images, even the same tissue may present diverse gray levels. Some images are visually brighter, while others are darker. In our ROI assignment framework, we define indicators $D_g$, $D_a$, and $D_s$ according to the relative grayscale or spatial information of regions in a single image, which contributes to the thresholds’ adaptation on dark or bright images. The three indicators are all $R_{on}$-dependent, and $D_s$ is also MAB-dependent. With the indicators’ variation under the update of $R_{on}$ and the MAB throughout the assignment procedure, some superior regional features are extracted in the image. To visually illustrate the effectiveness of these indicators, we recorded the $D_g$, $D_a$, and $D_s$ of the ROIs in 30 frames from Dataset I, as well as their classifications (calcification, adventitia, soft plaque, lumen, or shadow). The statistical results are displayed graphically in Figure 10. It is obvious that the components are distributed at different locations in the metric space formed by the three indicators. The thresholds controlling the assignment, $T_{a1}~T_{a4}$, $T_s$, and $T_{g1}~T_{g4}$, are determined according to the statistical results. Furthermore, to reduce assignment error, two sets of conditions are considered for calcification, adventitia, and soft plaque. One decision is more rigorous in terms of the intensity ($D_g$, $D_a$), while the other is more rigorous in terms of the location ($D_s$), as illustrated in Section 3.3.3 ① ② ③. Note that the discrimination error between calcification and soft plaque does not affect the extraction of the boundary.

Important parameters involved in SFCME-LSE are listed in Table 2. $D_c$ determines a circular catheter zone $R_c$ including the ring-down artifact and is usually 43 pixels in 40 MHz IVUS images. As guidewire artifacts touch $R_c$, we believe that the maximum distance between them is $T_d$. $L_{out}$ is set to 23 pixels, ensuring that pixels within this thickness belong to the shadow left by the calcification. $T_{\theta }$ is set to stop the update of the MAB, and $T_{area}$ is set to terminate the assignment of the ROIs. By changing values of weights $\mu$, $\beta$, and $\mathsf{\nu }$, the contribution of each term in Equation (17) to the evolution of deformable contours will be altered. We carried out several tests and chose the values $\mu =0.2$, $\beta =1$, and $\mathsf{\nu }=\pm 0.03$, yielding acceptable results for both borders.

The highlight of the SFCME-LSE method lies in its effective utilization of pathological structures and simple implementation. Impediments in IVUS image segmentation are utilized to provide topological constraints on the boundary. Compared to the time-consuming extraction of features and training of classifiers [3,4,5,6,7,8], unsupervised clustering with post-assignment is fast and requires no labeled data. The iterative update of indicators $D_g$, $D_a$, and $D_s$ can quickly mine valuable information in the image. Moreover, the results in Section 4 indicate the adaptability of the proposed framework to different datasets. Inevitably, there exist several limitations in the proposed method. The inaccuracy in the detection of the MAB derives from the loss of contour information in the shadow area. Segments obtained from radial linear interpolation may not coincide with the ground truth well, as shown in Figure 11 ①. Hypoechoic or echoless soft plaques may contain large amounts of lipids and necrotic tissue and appear in the IVUS image as areas of low intensity. Some soft plaques are significantly similar to the lumen, causing the segmentation error of the LIB, as depicted in Figure 11 ②. In the mentioned circumstances, the clustering and LSE techniques cannot accurately locate the boundary with indistinguishable intensities and textures on both sides. Future studies can be conducted to solve these problems by deep learning architectures or optical coherence tomography (OCT)-guided IVUS image segmentation, if there are enough annotated data.

6. Conclusions

In this paper, we propose the SFCME-LSE method for MAB and LIB detection in IVUS images. Superpixel-wise fuzzy clustering is robust to speckle noise in ultrasound images. A novel ROI assignment framework is proposed according to the anatomical and grayscale information of the image. The contour deforms towards the desired boundary through LSE with a new edge indicator. Experiments employed on two datasets demonstrate the effectiveness of the method on 40 MHz IVUS images. The statistical results of our method compared with other recently published methods on the publicly available dataset indicate the competitiveness of the SFCME-LSE method.