An artificial neural network method for lumen and media-adventitia border detection in IVUS

doi:10.1016/j.compmedimag.2016.11.003

Computerized Medical Imaging and Graphics

Volume 57, April 2017, Pages 29-39

https://doi.org/10.1016/j.compmedimag.2016.11.003 Get rights and content

Highlights

•
Used the artificial neural network (ANN) method as the feature learning algorithm, we got accurate results with simple feature input.
•
The error of our approach is close to the error of two manual trace results from two IVUS experts.
•
We did the Leave-one-out cross-validation, which proved the stability and adaptability of our approach.

Abstract

Intravascular ultrasound (IVUS) has been well recognized as one powerful imaging technique to evaluate the stenosis inside the coronary arteries. The detection of lumen border and media-adventitia (MA) border in IVUS images is the key procedure to determine the plaque burden inside the coronary arteries, but this detection could be burdensome to the doctor because of large volume of the IVUS images. In this paper, we use the artificial neural network (ANN) method as the feature learning algorithm for the detection of the lumen and MA borders in IVUS images. Two types of imaging information including spatial, neighboring features were used as the input data to the ANN method, and then the different vascular layers were distinguished accordingly through two sparse auto-encoders and one softmax classifier. Another ANN was used to optimize the result of the first network. In the end, the active contour model was applied to smooth the lumen and MA borders detected by the ANN method. The performance of our approach was compared with the manual drawing method performed by two IVUS experts on 461 IVUS images from four subjects. Results showed that our approach had a high correlation and good agreement with the manual drawing results. The detection error of the ANN method close to the error between two groups of manual drawing result. All these results indicated that our proposed approach could efficiently and accurately handle the detection of lumen and MA borders in the IVUS images.

Introduction

Atherosclerosis is a disease of the vessel wall, and it is known as the major cause of cardiovascular diseases such as heart attack or stroke (Frostegard, 2005). Stenosis in coronary artery caused by the atherosclerosis might lead to clinical complications like angina, myocardial infarction, and even sudden cardiac death. Cardiovascular diseases are receiving more attention because they account for 30% of all deaths worldwide (Morabia, 2005). Intravascular Ultrasound (IVUS) technique has been widely applied for diagnosing the arteriosclerotic disease of coronary artery in this decade (Brunenberg et al., 2006) because of its capability to visualize the inner structure of blood vessel in real time (Ciompi et al., 2009, Ciompo et al., 2011). IVUS allows monitoring and quantifying the state of the vessel wall and lumen and it is an intra-operative imaging tool for the quantification and characterization of coronary plaque, used for diagnostic purposes and for guiding Percutaneous Coronary Intervention (PCI), enabling the visualization of high resolution images of internal vascular structures. Compared with other angiographic imaging modalities, (e.g. X-ray, MRA and CT), IVUS enables the visualization of both vessel morphology and plaque. It provides extremely high image resolution (up to 113 um), which make it become the only modality enabling the accurate morphological segmentation of both vessel membranes (lumen and media) and the assessment of the plaque type in vivo (Nair et al., 2002, Sathyaranayana et al., 2009). An important task in the IVUS diagnosis is to determine the plaque burden, which requires the location of lumen border and the media-adventitia (MA) borders of blood vessel (Mendizabal-Ruiz et al., 2009). It is of substantial clinical interest and contributes to clinical decision making. Yet, no truly reliable and consistently accurate IVUS segmentation methods exist that would guarantee segmentation success in a clinical setting. Because of high volume of IVUS images, it could be very challenging to process the IVUS image in real time if the medical physicians need to identify these two borders by hand. Furthermore, high inter-observer and intra observer variability (up to 20%) might occur in the medical physicians without enough experiences (Meier et al., 1997). Therefore, it is a great demand to develop the computer- aided approach to detect the lumen and MA borders for efficient diagnostic and treatment of arteriosclerotic disease. Automated lumen segmentation of IVUS sequences has been a topic of interest since the early 1990s. Some approaches were based on the use of local properties of the image such as pixel intensity and gradient information (edges) combined with computational methods including graph search (Zhang et al., 1998, Sonka et al., 1995), active surfaces (Klingensmith et al., 2000), active contours (Kovalski et al., 2000), and in other approaches, segmentation was accomplished by the use of gray level variances to model ultrasound speckle (Luo et al., 2003), contrast of regions (Hui-Zhu et al., 2002), statistical properties of the image (Brusseau and de Korte, 2004, Gil et al., 2006), spatiotemporal information (3D segmentation) (Roy Cardinal et al., 2006), and discrete wavelet decomposition (dos Santos Filho et al., 2006). Most of previous attempts to detect the lumen and MA borders could be divided into three categories: active contour-based method, edge-based method, and probability-based method. The snake and the level set were two commonly used active contour-based methods for border detection (Katouzian et al., 2012), however, the contour evolution was heavily depend on the prior shape constraints of blood vessel. In the contrast, the edge-based method detects the borders by analyzing the appearance of edges in the IVUS images (Essa et al., 2011). The region-based method detects the borders by distinguishing the lumen region, plaque region and adventitia region using appearances difference among the three regions (Mendizabal-Ruiz and Kakadiaris, 2012). Nevertheless, the performance of these three methods highly relied on sophisticated features that might only work for specific problem. For example, in the active contour-based method, gradient vector flow is usually to deal with the problem of boundary concavities (Xu and Prince, 1998). For the edge-based method and region-based method, it is challenging to design proper edge features or region features in order to recognize the lumen and MA interfaces. It is of great demand to detect the lumen and MA borders using general imaging features, such as pixel and gradient information. In this work, we apply the artificial neural network (ANN) method to learn the hidden structures and connections of general imaging features and transform them into a more efficient representation. We use two types of data including spatial information, neighborhood pixels information as the input to the ANN. Our approach has been tested on a set of 461 IVUS images from four subjects by comparing to the manual drawing method.

Section snippets

Methodology

As one popular algorithm in the classification and regression problems, the ANN method can extract the hidden structures and connections among these image features by learning their compressed representation. In this study, we applied the ANN method to learn imaging features in order to detect the lumen and media-adventitia borders in IVUS images (Hilton and Salakhutdinov, 2006). After learning imaging features of different vessel layers using the ANN method, we could identify the borders by

Experiments and results

In order to validate the performance of our approach, our approach was compared to the manual drawing method on 461 IVUS images from four subjects. Fig. 5 displayed some example results of deteting the lumen and media-adventitia borders. The following content will explain the details of all the experiments in this work.

Data Collection: All the IVUS data were collected using a commercially available IVUS machine (Inision Gold, Volcano) with a 20 MHz solid-state IVUS catheter (EagleEye Volcano).

Discussion

Our method has a high accuracy. As shown in Table 1, the LCSA Correlation of Testing data is 99.047%, and manual compare value is 98.42%. The VCSA Correlation of Testing data is 96.48%, and manual compare value is 98.19%. The result is very close, which means the error between our approach and the manual trace, is closed to the error between the two manual trace results from the two IVUS experts. The PCSA Correlation of Testing data is 86.479%, and manual compare value is 94.253%. PCSA is more

Conclusion

In this study, we developed a double ANN approach to detect the lumen and MA borders from IVUS images. Unlike other methods (Yan et al., 2014, Sofian et al., 2015), our method does not require any special human feature extraction and calibration. It has extremely simple initialization, strong adaptability, good robustness, and high accuracy. The snak process play a role in the smoothing curve. The proposed approach has been evaluated on 461 IVUS images acquired from four subjects by different

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant 81271645, Guangdong Image-guided Therapy Innovation Team (2011S013), and the national Key research and Development Program of China (2016YFC1300300).

References (34)

G. Kovalski et al.
Three-dimensional automatic quan-titative analysis of intravascular ultrasound images
Ultrasound Med. Biol.
(2000)
D. Meier et al.
Automated morphometry of coronary arteries with digital image analysis of intravascular ultrasound
Am. Heart J.
(1997)
G. Mintz et al.
American College of Cardiology clinical expert consensus document on standards for acquisition, measurement and reporting of intravascular ultrasound studies (ivus): a report of the american college of cardiology task force on clinical expert consensus documents developed in collaboration with the european society of cardiology endorsed by the society of cardiac angiography and interventions
J. Am. Coll. Cardiol.
(2001)
M. Alireza et al.
K-Sparse autoencoders
International Conference on Learning Representations, ICLR
(2014)
S. Balocco et al.
Standardized evaluation methodology andreference database for evaluating IVUS image segmentation
ComputMed. Imaging
(2013)
E. Brunenberg et al.
Automatic IVUS segmentation of atherosclerotic plaque with stop & go snake
International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)
(2006)
E. Brusseau et al.
Fully automatic luminal contour segmentation in intra-coronary ultrasound imaging—a statistical approach
IEEE Trans. Med. Imaging
(2004)
F. Ciompi et al.
ECOC random fields for lumen segmentation in radial artery IVUS sequences
International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)
(2009)
F. Ciompo et al.
A holistic approach for the detection of media-adventitia border in IVUS
International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)
(2011)
B. Efron et al.
The jackknife estimate of variance
Ann. Stat.
(1981)

E. Essa et al.

Automatic IVUS media-adventitia bor-der extraction using double interface graph cut segmentation

IEEE International Conference on Image Processing (ICIP)

(2011)

J. Frostegard

Sle, atherosclerosis and cardiovascular disease

J. Intern. Med.

(2005)

D. Gil et al.

Statistical strat-egy for anisotropic adventitia modelling in IVUS

IEEE Trans. Med. Imaging

(2006)

G. Hilton et al.

Reducing the dimensionaloty of data with neural networks

Science

(2006)

H. Hui-Zhu et al.

IVUS image segmentation based on contrast

Proc SPIE Medical Imaging: Image Processing

(2002)

M. Kass et al.

Snakes: active contour models

Int. J. Comput. Vision

(1988)

A. Katouzian et al.

A state-of- the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images

IEEE Trans. Inf. Technol. Biomed.

(2012)

Cited by (59)

A transformer-based pyramid network for coronary calcified plaque segmentation in intravascular optical coherence tomography images
2024, Computerized Medical Imaging and Graphics
Characterizing coronary calcified plaque (CCP) provides essential insight into diagnosis and treatment of atherosclerosis. Intravascular optical coherence tomography (OCT) offers significant advantages for detecting CCP and even automated segmentation with recent advances in deep learning techniques. Most of current methods have achieved promising results by adopting existing convolution neural networks (CNNs) in computer vision domain. However, their performance can be detrimentally affected by unseen plaque patterns and artifacts due to inherent limitation of CNNs in contextual reasoning. To overcome this obstacle, we proposed a Transformer-based pyramid network called AFS-TPNet for robust, end-to-end segmentation of CCP from OCT images. Its encoder is built upon CSWin Transformer architecture, allowing for better perceptual understanding of calcified arteries at a higher semantic level. Specifically, an augmented feature split (AFS) module and residual convolutional position encoding (RCPE) mechanism are designed to effectively enhance the capability of Transformer in capturing both fine-grained features and global contexts. Extensive experiments showed that AFS-TPNet trained using Lovasz Loss achieved superior performance in segmentation CCP under various contexts, surpassing prior state-of-the-art CNN and Transformer architectures by more than 6.58% intersection over union (IoU) score. The application of this promising method to extract CCP features is expected to enhance clinical intervention and translational research using OCT.
Deep learning segmentation, classification, and risk prediction of complex vascular lesions on intravascular ultrasound images
2023, Biomedical Signal Processing and Control
This study aimed to access the performance of our deep learning-based model in the segmentation and classification of vascular lesions on IVUS images.
A total of 5,089 IVUS frames derived from 100 patients with stable or unstable angina pectoris were retrospectively collected. Our deep learning diagnostic framework composed of two stages, image segmentation and lesion classification. Segmentation was performed using the dilated attention U-Net model. The standard classifier, ResNet18, was subsequently used to classify the lesions into six groups, plaques (fibrous, lipid, and calcified), dissections, hematomas, and thrombi. Segmentation performance was evaluated in terms of Dice similarity coefficient (DSC) and 95% Hausdorff distance (HD95), while classification was assessed based on sensitivity, specificity, area under the receiver operating characteristic curve, and F1 score. The predictive value of our proposed model for high-risk lesions based on geometric measurements was further assessed.
The segmentation performance of our model was deemed satisfactory, with average DSC values of 80.75%, 86.68%, and 79.21% demonstrated for dissections, hematomas, and thrombi, respectively. Good lesion classification performance was observed as well, with F1 scores of 94.89%, 95.91% and 96.42%, respectively. Our model further demonstrated the ability to stratify lesions at risk for dissection.
Our deep-learning diagnostic framework demonstrated accuracy in the identification, classification, and risk stratification of vascular lesions based on IVUS images. It is clinically conducive, easily adoptable, and enables the early diagnosis of complex lesions at risk of major adverse cardiovascular events.
Analysis methods of coronary artery intravascular images: A review
2022, Neurocomputing
Citation Excerpt :
They applied support vector machine as the classifier and achieved outstanding lumen segmentation performance. Su, et al. [55], Su, et al. [56] adopted an artificial neural network (ANN) which consists of two sparse auto-encoders differentiating vascular layers to segment lumen contour in intravascular ultrasound images based on spatial, neighboring, and gradient features. Their neural network proved its effectiveness through experimental results.
Coronary artery disease is among one of the diseases human suffer most. Intravascular coronary arterial image analysis consists of denoising, segmentation, detection, and three-dimensional reconstruction, having a significant meaning for auxiliary diagnosis and treatment of coronary artery disease. Intravascular ultrasound (IVUS) and intravascular optical coherence tomography (IVOCT) are the two most commonly applied intravascular coronary arterial imaging techniques. Based on these fundamental imaging techniques, in recent years, many advanced technologies from traditional machine learning algorithms to deep learning methods were employed in the analysis of intravascular coronary arterial images and made huge progress in this field. In this survey, we reviewed more than one hundred papers published in top journals or conferences such as Neural Networks and MICCAI. These papers proposed approaches or schemes for the intravascular coronary arterial image analysis, including lumen border segmentation, atherosclerotic plaque characterization, media-adventitia segmentation, stent strut detection, and three-dimensional reconstruction. Our survey began with introducing coronary artery intravascular imaging techniques, essential neural networks, and deep learning and then presented an across-the-board review of methods, applications, and trends of intravascular image analysis. This survey is more comprehensive than other articles not only for its scope and reference number but also for discussing the future direction in this field. Compared to other review papers in this field, this article could assist beginners in constructing a basic knowledge frame of coronary artery intravascular image analysis methods and brought state-of-the-art progress in this field to fellow researchers. We hope this paper could benefit either the beginners for coronary arterial image analysis or experienced researchers.
Convolutional networks for the segmentation of intravascular ultrasound images: Evaluation on a multicenter dataset
2022, Computer Methods and Programs in Biomedicine
The delineation of the lumen contour and external elastic lamina (EEL) in intravascular ultrasound (IVUS) images is crucial for the quantitative analysis of coronary atherosclerotic plaques. However, the presence of ultrasonic shadows and anatomical structures (such as bifurcations and calcified plaque) complicates the automatic delineation of the lumen contour and EEL. The purpose of this paper is to evaluate the IVUS segmentation performances of different convolutional networks and the impact factors on a large-scale multiple-center dataset.
A total of 6516 cross-sectional images from 175 IVUS pullbacks acquired in different centers by different IVUS imaging catheters were screened from a corelab to evaluate the segmentation methods. The IVUS images included bifurcation, side branch ostia, and various image artifacts to reflect the general image characteristics in routine clinical acquisition. We compared three generic fully convolutional networks (FCNs) and two FCNs specifically designed for the segmentation of IVUS images and explored the factors impacting the segmentation performance, including the training images and the input of consecutive images to the models. The performance of the FCNs was evaluated by using the Dice similarity coefficient (DSC), the Jaccard index (JI), the Hausdorff distance (HD), linear regression and Bland-Altman analysis.
The 4-cascaded RefineNet and DeepLabv3+ outperformed U-net and IVUS-net in the segmentation of the lumen contour and EEL on IVUS images. DeepLabv3+ had the best segmentation performance, with DSCs of 0.927 and 0.944, JIs of 0.911 and 0.933, and HDs of 0.336 mm and 0.367 mm for delineation of the lumen and EEL, respectively. Excellent agreement between DeepLabv3+ and the manual delineation was found in the quantification of the coronary plaque area (r = 0.98).
The convolutional network architecture is effective in the automatic segmentation of IVUS images. It might contribute to the clinical application of quantitative IVUS analysis in real-world as well as the efficient assessment of coronary atherosclerosis.
Advanced deep learning methodology for accurate, real-time segmentation of high-resolution intravascular ultrasound images
2021, International Journal of Cardiology
Citation Excerpt :
Nevertheless the applications of DL-methodologies for the processing of IVUS imaging data are limited. [8–17] This is due to the small number of IVUS annotations used for training that did not allow full development of the model, [8–13] methodologies not being designed to segment both EEM and lumen borders thus not being able to analyse the plaque burden and volume, [17] the limited efficacy of the proposed methods in accurately detecting the lumen border, [14,16] the inability of the existing methods to process challenging IVUS frames with side branches and increased calcific burden and to the fact that the developed algorithms have not been tested in large datasets so as to have objective evidence about their efficacy and potential value in clinical practice. [8–13]. In the present study we introduce an advanced DL-based segmentation method which appears able to overcome the above limitations.
The aim of this study is to develop and validate a deep learning (DL) methodology capable of automated and accurate segmentation of intravascular ultrasound (IVUS) image sequences in real-time.
IVUS segmentation was performed by two experts who manually annotated the external elastic membrane (EEM) and lumen borders in the end-diastolic frames of 197 IVUS sequences portraying the native coronary arteries of 65 patients. The IVUS sequences of 177 randomly-selected vessels were used to train and optimise a novel DL model for the segmentation of IVUS images. Validation of the developed methodology was performed in 20 vessels using the estimations of two expert analysts as the reference standard.
The mean difference for the EEM, lumen and plaque area between the DL-methodology and the analysts was ≤0.23mm² (standard deviation ≤0.85mm²), while the Hausdorff and mean distance differences for the EEM and lumen borders was ≤0.19 mm (standard deviation≤0.17 mm). The agreement between DL and experts was similar to experts' agreement (Williams Index ranges: 0.754–1.061) with similar results in frames portraying calcific plaques or side branches.
The developed DL-methodology appears accurate and capable of segmenting high-resolution real-world IVUS datasets. These features are expected to facilitate its broad adoption and enhance the applications of IVUS in clinical practice and research.
Improved real-time segmentation of Intravascular Ultrasound images using coordinate-aware fully convolutional networks
2021, Computerized Medical Imaging and Graphics
Citation Excerpt :
Firstly, we train two separate single-class binary segmentation models for the Lumen and Media. This mirrors the approach taken in previous work (Su et al., 2017; Yang et al., 2018) with separate networks trained for extraction of the Lumen and Media. We evaluate our architectural improvements in this binary setting.
Segmentation of Intravascular Ultrasound (IVUS) images into Lumen and Media (interior and exterior) artery vessel walls is highly clinically relevant in the diagnosis and treatment of cardiovascular diseases such as atherosclerosis. When fused with position data, such segmentations also play a key role in reconstructing 3D representations of arteries. Automated segmentation in real-time is known to be a difficult image analysis problem, primarily due to artefacts commonly present in IVUS ultrasound images such as shadows, guide-wire effects, and side-branches. An additional challenge is the limited amount of expert labelled IVUS data, which limits the application of many well-performing deep learning models from other domains. To exploit the circular layered structure of the artery in B-Mode images, we propose a multi-class fully convolutional semantic segmentation network based on a minimal U-Net architecture augmented with learned translation dependence in the polar domain. The coordinate awareness in the multi-class segmentation allows the model to exploit relative spatial context about the interior and exterior vessel walls which are simply separable in polar coordinates. After training on 109 expert-labelled examples, our model significantly outperforms the state-of-the art in terms of mean Jaccard Measure (0.91 vs. 0.89) and Hausdorff distance (0.32 mm vs. 0.48 mm) on Media segmentation, and reaches equivalent performance on Lumen segmentation when evaluated on a standard publicly available dataset of 326 IVUS B-Mode images captured by 20 Mhz ultrasound probes. Using an order of magnitude fewer trainable parameters than the previous state-of-the-art, our model runs over 50 times faster and is able to execute in only 3 ms on a common GPU, achieving both leading accuracy and practical real-time performance.

View all citing articles on Scopus

¹: Shengran Su and Zhenghui Hu contributed equally to the writing of this article.

View full text

An artificial neural network method for lumen and media-adventitia border detection in IVUS

Highlights

Abstract

Introduction

Section snippets

Methodology

Experiments and results

Discussion

Conclusion

Acknowledgments

Ultrasound Med. Biol.

Am. Heart J.

J. Am. Coll. Cardiol.

K-Sparse autoencoders

International Conference on Learning Representations, ICLR

Standardized evaluation methodology andreference database for evaluating IVUS image segmentation

ComputMed. Imaging

Automatic IVUS segmentation of atherosclerotic plaque with stop & go snake

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)

Fully automatic luminal contour segmentation in intra-coronary ultrasound imaging—a statistical approach

IEEE Trans. Med. Imaging

ECOC random fields for lumen segmentation in radial artery IVUS sequences

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)

A holistic approach for the detection of media-adventitia border in IVUS

International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI)

The jackknife estimate of variance

Ann. Stat.

Automatic IVUS media-adventitia bor-der extraction using double interface graph cut segmentation

IEEE International Conference on Image Processing (ICIP)

Sle, atherosclerosis and cardiovascular disease

J. Intern. Med.

Statistical strat-egy for anisotropic adventitia modelling in IVUS

IEEE Trans. Med. Imaging

Reducing the dimensionaloty of data with neural networks

Science

IVUS image segmentation based on contrast

Proc SPIE Medical Imaging: Image Processing

Snakes: active contour models

Int. J. Comput. Vision

A state-of- the-art review on segmentation algorithms in intravascular ultrasound (IVUS) images

IEEE Trans. Inf. Technol. Biomed.