Detection of microaneurysms using multi-scale correlation coefficients
Introduction
Images of the ocular fundus, also known as retina images or retinal (fundus) images, can provide useful information about retinal [1], ophthalmic, and even systemic diseases such as diabetes [2], [3], [4], [5], [6], [7], [8], [9], [10], [11], [12], [13], [14], hypertension [15], glaucoma [16], [17], obesity arteriosclerosis [18] and retinal artery occlusion. One such condition, diabetic retinopathy (DR), is the result of long-term diabetes and involves the formation on the retina of lesions which can lead to blindness.
In order to prevent the damage of this severe complication to patients’ vision, it is very important to diagnose diabetic retinopathy and provide appropriate treatment to minimize further deterioration as early as possible. Since microaneurysms are the first signs of DR, its detection is vital. It is also crucial to monitor the development of the disease and classify changes in retinal images taken at different medical examinations to evaluate the effectiveness of the medical treatment and observe the evolution of DR.
In general, DR can be classified into four stages: mild non-proliferative diabetic retinopathy (NPDR), moderate NPDR, severe NPDR, and proliferative diabetic retinopathy (PDR). The damage caused by DR can be reduced and major vision loss [19], [20] prevented if it is diagnosed and treated in its early stages. Thus, regular examination of diabetic patients’ retina is very important. However, it is time consuming and subject to human errors if DR diagnosis is conducted by medical professionals manually. Therefore, automated or computer-aided analysis of diabetic patients’ retina can help eye care specialist to screen larger populations of patients more accurately. An image processing approach provides a powerful tool in three aspects: image enhancement and feature extraction (feature based image registration), mass screening (diagnosis) and change detection and classification (monitoring).
Fluorescein angiography (FA) is an imaging technique for evaluation of retinal vascular disease, particularly DR [7], [8], [9], [14], [21]. Fluorescein dye is injected intravenously and the fluorescence within retinal vessels is then photographed through a matched combination of excitation and barrier filters. Although FA produces very clear gray-scale retinal images as seen in Fig. 1(a) and is effective for describing hemorrhages and neovascularization, it is not well-accepted by patients because of its intrusive nature. Therefore, it is essential to develop a safe, fast, easy, and comfortable way to observe and capture the retina. The analysis of color retinal images [3], [4], [5], [6], [13] seen in Fig. 1(b) (a color fundus image, produced by a fundus camera) is viewed as this feasible approach because the acquisition of color retinal images is non-intrusive, very fast and easy. A fundus camera is essentially a specialized microscope with an attached camera that allows you take photographs of the interior surface of the eye.
Processing of color retinal images is usually conducted in its green channel since microaneurysms have the highest contrast with its background here. This is illustrated in Fig. 2, which represents the image components in three different color bands—Fig. 2(a) is an original color retinal image, Fig. 2(b–d) represents component in red band (), green band () and blue band (), respectively. In the green band objects such as blood vessels, microaneurysms, the optic disk, etc., are most visible.
In [11], an artificial neural network (ANN) was used to automatically classify microaneurysms. The ANN consisted of an input pixel layer, hidden layer and output layer, and was trained to recognize features such as blood vessels, exudates and hemorrhages (hemorrhages or microaneurysms). Each image was divided into 30×30 and 20×20 squares pixels depending on the feature being detected and a trained observer then classified the squares as normal retina not showing blood vessels (normal), normal retina showing normal blood vessels (vessel), retina showing exudates (exudates), or retina showing hemorrhages or microaneurysms (hemorrhage). However, this method only classifies the regions of microaneurysms without extraction and localization. In addition, the use of a neural network is time consuming.
It is very difficult to detect microaneurysms by classifying hemorrhages because their pixel values are similar to that of blood vessels. Several approaches have been proposed to tackle this challenge. In [13], a mathematical morphology [14] based method was developed to detect microaneurysms in color retinal images. The proposed algorithm consisted of several stages. In the image preprocessing stage the image background was removed to create a “shade corrected” image. This was accomplished by subtracting the image with the result of a 25×25 median filter applied to the image. Afterwards, candidate extraction was performed on the “shade corrected” image by extracting vessels with morphological opening using 12 rotated linear structuring elements at 15° and produced 12 responses. Taking the maximum pixel values of all 12 responses at each pixel location produced a vascular map which was subsequently removed from the “shade corrected” image. The remaining candidates inside this image should no longer contain elongated structures but red lesions. Region growing was applied to each candidate before 13 features based on shape and color intensities were extracted. This was used to construct a feature vector for classification. The author also described another way to detect red lesions in fundus images based on the fusion of candidate detection and pixel classification. Initially, all of the lesion candidates were located by combining mathematical morphology [14] with pixel classification, where the pixels were classified using a k-NN classifier with a reference standard that requires manually marking each image pixel. Finally, all of the marked candidates are further classified as red lesion or non-red lesion by a k-NN using additional 68 features extracted from each candidate.
It is noted that pixel classification found in [11], [13] involves medical experts labeling each pixel, something not feasible when using greater number of images. When the length of the structuring element [13], [14] is increased to be able to detect larger objects, the vessel segmentation deteriorates leading to more spurious candidate objects being detected on the vessel, which limits the effectiveness of the proposed methods.
The algorithm presented in [28] applied wavelet image decomposition. It compared a small window of the image with a microaneurysm template modeled by a Gaussian curve. The comparative result was the squared errors sum between coefficients of each sub-band and the microaneurysm's corresponding coefficients. Thresholds were used to separate microaneurysms from other structures. However, only two scales were used to model microaneurysms, which is not sufficient for real applications.
A three-stage scheme for the detection of microaneurysms was presented in [25]. Firstly, the image was divided into different sub-regions with image enhancement. Secondly, local adaptive thresholding was applied to detect microaneurysms. Thirdly, prior knowledge in the sense that microaneurysms cannot occur on the optical disk, vessels and hard exudates were incorporated for final output. Unfortunately, a comprehensive testing on the proposed algorithm is not reported except the preliminary results on the detection of additional objects such as optical disk and hard exudates in retinal images.
There are many other methods for the detection of microaneurysms. The algorithm presented in [27] can be divided into four steps. The first step consisted of image enhancement: shade correction and image normalization of the green channel. The second step was candidate detection using diameter closing with an automatic threshold scheme. The third step involved feature extraction to facilitate the forth step of classification which was based on kernel density estimation in conjunction with Bayesian risk minimization. However, the diameter closing and thresholding in the second step may cause the loss of important feature points which are crucial for further processing and final classification.
Local contrast normalization with local vessel detection was regarded as another useful approach to detect microaneurysms [26]. After the initial preprocessing stage, a watershed retinal region growing method was applied to be used for contrast normalization of each candidate microaneurysm. Local vessel detection that occur with microaneurysms and a k-NN classifier was used to classify the candidates. Nevertheless, this method is complicated and its results are based on cross-validation of the training set instead of splitting the dataset into training and test.
To avoid the limitations of the above algorithms for better performance, this paper presents a novel approach to the CAD of diabetic retinopathy that applies a hierarchical approach to detect microaneurysms in retinal images. The approach applies multi-scale correlation filtering (MSCF) and dynamic thresholding for intensity-based detection and localization of microaneurysms in retinal images in a two-level hierarchical architecture. In the first level (coarse level), we detect candidate microaneurysms using MSCF. In the second level (fine level), we classify true microaneurysms by extracting 31 features from the level one candidates which are used to classify them. Fig. 3 shows the architecture of the proposed system.
The remainder of this paper is organized as follows. 2 Coarse level: microaneurysm candidate detection, 3 Fine level: true microaneurysm classification describe the two level system architectures in detail. Section 4 presents and discusses the experimental results. Section 5 offers our conclusion and an outline of future work.
Section snippets
Coarse level: microaneurysm candidate detection
The task for course-level detection is to identify all possible microaneurysm candidates in a retinal image. Fig. 4(a) and (b) shows two microaneurysms that were found in Fig. 1(b) (shown in its green channel) and Fig. 4(c) and (d) shows their corresponding grayscale distributions. As can be seen, microaneurysms exhibit a Gaussian shape. This allows us to use a Gaussian function to detect microaneurysms according to the similarity between the distributions of its grayscale. The Gaussian
Fine level: true microaneurysm classification
The task for fine level microaneurysm classification is to detect true microaneurysms in the candidate set, which can be implemented through feature extraction. We used a total of 31 features for each candidate, based on shape, grayscale pixel intensity, color intensity, responses of Gaussian filter-banks, and correlation coefficient values. Table 1 lists the 31 features used to discriminate microaneurysms in our proposed approach. Features 27–29 are unique to the proposed method as it is based
Experimental results and analysis
The experimental results reported in this paper are mainly based on our work submitted to retinopathy on-line challenge (ROC), an international competition associated with 2009 SPIE medical imaging (MI’ 2009). The score of our submission was ranked second among all participants. The details of ROC competition are available on ROC website [22]. A preliminary performance evaluation was also conducted on the well established public dataset DIARETDB1 (standard diabetic retinopathy database) [23],
Conclusion and future work
In this paper we proposed a hierarchical approach based on multi-scale correlation filtering (MSCF) to detect all microaneurysms from a color retinal image. This consisted of coarse level: microaneurysm candidate detection using MSCF and fine level: true microaneurysm classification. The approach was evaluated extensively using the public retinal image database provided on the ROC competition website [22] as part of our participation in this event. We conclude that the proposed approach is
Acknowledgments
The authors are most grateful for the constructive advice on the revision of the manuscript from the anonymous reviewers. The funding support from Hong Kong Government under its GRF scheme and the Research Grant from Hong Kong Polytechnic University are greatly appreciated.
About the Author—BOB ZHANG graduated from York University with B.Sc. in Computer Science in 2005 and obtained his M.Sc. from Concordia University, Canada in 2007. Currently he is a Ph.D. candidate in Department of Electrical and Computer Engineering, the University of Waterloo, Canada. His research interest includes pattern recognition, medical imaging and machine learning.
References (29)
- et al.
A comparison of computer based classification methods applied to the detection of microaneurysms in opthalmic fluorescein angiograms
Comput. Biomed. Res.
(1998) - et al.
An image-processing strategy for the segmentation of microaneurysms in fluorescein angiograms of the ocular fundus
Comput. Biomed. Res.
(1996) - et al.
Retinal vessel diameter and open-angle glaucoma: the Blue Mountains eye study
Ophthalmology
(2005) - et al.
Automated segmentation of the optic nerve head for diagnosis of glaucoma
IEEE Trans. Med. Image Anal.
(2005) - et al.
Irregular motion recovery in fluorescein angiograms
Pattern Recognition Lett.
(1997) - et al.
Automatic detection of microaneurysms in color fundus images
Med. Image Anal.
(2007) - M. Goldbaum, S. Moezzi, A. Taylor, S. Chatterjee, J. Boyd, E. Hunter, R. Jain, Automated diagnosis and image...
- et al.
A fully automated comparative microaneurysm digital detection system
Eye
(1997) - et al.
Automated detection of microaneurysms in digital red-free photographs: a diabetic retinopathy screening tool
Diabetic Med.
(2000) - et al.
Automated detection of fundus photographic red lesions in diabetic retinopathy
Invest. Ophthalmol. Vis. Sci.
(2003)
A contribution of image processing to the diagnosis of diabetic retinopathy—detection of exudates in color fundus images of the human retina
IEEE Trans. Med. Imaging
Automated detection of diabetic retinopathy on digital fundus images
Diabetic Med.
Automatic segmentation of microaneurysms in retinal angiograms of diabetic patients
Proc. IEEE Int. Conf. Image Anal. Appl.
Automated detection and quantification of microaneurysms in fluorescein angiograms
Graefe's Arch. Clin. Exp. Ophthalmol.
Cited by (222)
Deep learning enabled hemorrhage detection in retina with DPFE and splat segmentation in fundus images
2024, Biomedical Signal Processing and ControlSpatiotemporal variation of water cycle components in Minjiang River Basin based on a correction method for evapotranspiration products
2023, Journal of Hydrology: Regional StudiesAutomatic detection of microaneurysms in fundus images based on multiple preprocessing fusion to extract features
2023, Biomedical Signal Processing and ControlAutomated lesion segmentation in fundus images with many-to-many reassembly of features
2023, Pattern RecognitionMicroaneurysms detection in retinal images using a multi-scale approach
2023, Biomedical Signal Processing and ControlCitation Excerpt :Furthermore, MAs vary in shape and size [17], being difficult to be fully characterized by morphological operations. Other approaches use template matching [11,15,20,22]. Zhang et al. [15] used a multi-scale Gaussian correlation filtering and dynamic thresholding.
About the Author—BOB ZHANG graduated from York University with B.Sc. in Computer Science in 2005 and obtained his M.Sc. from Concordia University, Canada in 2007. Currently he is a Ph.D. candidate in Department of Electrical and Computer Engineering, the University of Waterloo, Canada. His research interest includes pattern recognition, medical imaging and machine learning.
About the Author—XIANGQIAN WU graduated from Harbin Institute of Technology (HIT), China with M.Sc. and Ph.D. in 2000 and 2004, respectively. Currently he is an associate professor in HIT with research interests in image processing and pattern recognition.
About the Author—JANE YOU obtained her B.Eng. in Electronic Engineering from Xi’an Jiaotong University in 1986 and Ph.D. in Computer Science from La Trobe University, Australia in 1992. She was a lecturer at the University of South Australia and senior lecturer at Griffith University from 1993 till 2002. Currently she is an associate professor at the Hong Kong Polytechnic University. Her research interests include image processing, pattern recognition, medical imaging, biometrics computing, multimedia systems and data mining.
About the Author—QIN LI graduated from Zhengzhou University with B.Sc. in Computer Science and obtained his M.Sc. in Computer Science from the University of Newcastle, UK. Currently he is a Ph.D. candidate at the Hong Kong Polytechnic University. His research area includes image processing, pattern recognition and multimedia systems.
About the Author—FAKHRI KARRAY is a professor in Department of Electrical and Computer Engineering, the University of Waterloo, Canada. His research interests include machine learning, data mining, computer systems.