Regularizer based on Euler characteristic for retinal blood vessel segmentation☆
Introduction
Diabetic retinopathy (DR) is one of the causes of visual impairment in aging population and signifies an increased risk of coronary heart disease [1]. As it involves changes in the shape and structure of blood vessels in the fundus, visual impairment is often detected by analyzing fundus images. Fundus images are color images that represent the inner surface of the human eye and are often used by doctors to study at the retina and the anatomical components of the eye. Therefore, accurate identification of large and small retinal vessels with diameters of less than a pixel can be used as an early bio-marker for the diagnosis of DR [2], [3].
However, fundus images are affected by noise. Moreover, examining the small blood vessels is difficult and time-consuming. Hence, automatic characterizing of retinal blood vessels is important for the detection of DR. To this end, computer-aided segmentation of retinal blood vessels, based on pattern recognition, and supervised and unsupervised machine learning, has been successfully proposed [4], [5], [6]. Various studies in recent times have reported improve performance of blood vessel segmentation based on deep learning techniques [7], [8], [9], [10], [11], [12], [13]. Several U-Net models based on convolutional neural networks (CNNs) have been developed by modifying the number of layers in the encoder and decoder architectures and have achieved better results for retinal blood vessel segmentation [9], [14], [15], [16]. However, these approaches considered large and medium vessels as it appears in the ground truth. They failed to characterize the damaged tiny or small vessels, and hence unreliable diagnosis may occur. In this study, we aimed to delineate, in addition to the thick vessels, damaged tiny or small vessels that do not appear in the ground truth but appear in the original fundus images.
To address the small vessel connectivity problem, authors in [17] introduced a graph-based smoothing (GS) regularizer that pushes the network to consider small vessel. The GS regularizer hold smoothness level by calculating graph laplacians between the vessel and surrounding background area. In contrast, in this study, we proposed a different approach to penalize the network to pay more attention to small vessels by utilizing the number of isolated objects, as shown in the Fig. 1. Furthermore, we observe that most of the existing architectures fail to accurately detect the vessel connections especially on the thinner and small branches of the vessels (Fig. 1c).
However, for accurate segmentation, it important to preserve the vanished blood vessels as isolated objects so as to use them for finding entire vessel connections. Hence, this study is attempted to trace the retinal vessel connections using Euler characteristics (EC). The EC is an essential topological invariant for realizing the number of components and number of enclosed cavities to utilize the EC as a demonstration of vessel connectivity [18], [19].
We introduced a regularizer based on EC to minimize the number of isolated objects, especially in the small branches and tiny retinal vessel regions. The proposed method is based on the criterion of a vessel structure that is fully connected or if consisting of minimal number of isolated objects.
Specifically, we proposed two regularizers, minimum number of isolated objects (MISO) and differences of number of isolated objects (DISO) between predicted and true segmentation groundtruth based on EC, to delineate tiny vessel regions. We developed a U-Net-like CNN architecture to evaluate the performance of the proposed regularizers in delineating small vessel connections. The interesting key point in this study is implementing mathematical topology approach in the neural network architecture. The main contribution of this study is summarized as follows : (i) We proposed EC-based regularizers to estimate the number of isolated objects, (ii) compared the architecture performance of the proposed MISO and DISO regularizers with the architecture without regularizer, (iii) compared the performance of our proposed regularizers with GS and combined GS plus isolated objects (GISO) regularizers in segmenting the vessel connections, and (iv) analyzed the performance of the proposed approach with the state-of-the-art method for recognizing small blood vessel regions.
Section snippets
Related works
Image segmentation plays an important role in medical imaging applications. In deep learning domain, segmentation area has been widely utilized to support medical image analysis [20], [21]. In [22], the authors developed a multi-label method based on supervised structured for the segmentation of retinal vessels. They applied pre-processing methods before feeding the image into the network. Ortiz et al. [7], in contrast, proposed a deep CNN that ignored pre-processing steps and directly used raw
Proposed methodology
This study proposed EC-based regularizers to estimate the number of isolated objects in U-Net-like deep CNN architecture for delineating small retinal vessel connections on a fundus image. Before training process, we enhanced the dataset with some pre-processing steps. The image is converted into grayscale followed by data normalization and contrast-limited adaptive histogram equalization (CLAHE). We applied normalization to keep the image into the same scale and CLAHE technique to enhance the
Dataset
The proposed network structure using EC regularizer for accurate vessel segmentation is evaluated on DRIVE, STARE, and CHASEDB1 fundus image datasets. The DRIVE dataset (Canon CR5 nonmydriatic 3CCD camera at 45 field of view) consisted of 40 images with the size pixels and 8 bits per color channel taken with the field of view of approximately 540 pixels in diameter [3]. The STARE dataset (TopCon TRV-50 fundus camera at 35 field of view) included 20 images with the size of
Results and discussion
The vessel regions with complicated vessel structure shows the intersection of the vessels with large and tiny vessels. It is shown with some representative examples in Fig. 5. It demonstrated that the U-Net-like network without specific regularizer showed poor ability in delineating the vessel with complicated structure. However, if incorporated with our proposed regularizer based on EC, it can accurately identify the detailed vessel junctions and tiny vessels. Table 1 presents the performance
Conclusion
This study proposed EC-based regularizers to estimate the number of isolated objects in U-Net-like deep CNN architecture for delineating small retinal vessel connections on a fundus image. The proposed EC-based MISO and DISO regularizers approach demonstrated improvement in retaining the width of the small and disjointed vessel connectivities through its high AUC values. Our approach can segment more number of vessels and can reconnect all isolated vessels, which is superior than the
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgments
This work was partly supported by JSPS KAKENHI grant number 16K00239 and 18F18112.
References (37)
- et al.
A hybrid method for blood vessel segmentation in images
Biocybern. Biomed. Eng.
(2019) - et al.
Blood vessel segmentation methodologies in retinal images–a survey
Comput. Methods Programs Biomed.
(2012) - et al.
Retinal vessel segmentation of color fundus images using multiscale convolutional neural network with an improved cross-entropy loss function
Neurocomputing
(2018) - et al.
Retinal vessel segmentation based on fully convolutional neural networks
Expert Syst. Appl.
(2018) - et al.
DUNet: a deformable network for retinal vessel segmentation
Knowl.-Based Syst.
(2019) - et al.
On the computation of the Euler number of a binary object
Pattern Recognit.
(1996) - et al.
A cross-modality learning approach for vessel segmentation in retinal images
IEEE Trans. Med. Imaging
(2016) Overview of epidemiologic studies of diabetic retinopathy
Ophthalmic Epidemiol.
(2007)- et al.
Addition of primary care-based retinal imaging technology to an existing eye care professional referral program increased the rate of surveillance and treatment of diabetic retinopathy
Diabetes Care
(2005) - et al.
Ridge-based vessel segmentation in color images of the retina
IEEE Trans. Med. Imaging
(2004)
A review of vessel extraction techniques and algorithms
ACM Comput. Surv. (CSUR)
Retinal blood vessel segmentation by multi-channel deep convolutional autoencoder
International Joint Conference SOCO’18-CISIS’18-ICEUTE’18
Retinal blood vessels semantic segmentation method based on modified U-Net
Proceedings of the 30th Chinese Control and Decision Conference, CCDC 2018
Joint segment-level and pixel-wise losses for deep learning based retinal vessel segmentation
IEEE Trans. Biomed. Eng.
Deep retinal image understanding
International Conference on Medical Image Computing and Computer-Assisted Intervention
DeepVessel: retinal vessel segmentation via deep learning and conditional random field
International Conference on Medical Image Computing and Computer-Assisted Intervention
A global and local enhanced residual U-Net for accurate retinal vessel segmentation
IEEE Comput. Archit. Lett.
U-Net: convolutional networks for biomedical image segmentation
International Conference on Medical Image Computing and Computer-Assisted Intervention
Cited by (0)
- ☆
Handle by Associate Editor Michele Nappi, Ph.D.