Abstract
Retinal fundus image analysis (RFIA) for diabetic retinopathy (DR) screening can be used to reduce the risk of blindness among diabetic patients. The RFIA screening programs help the ophthalmologists to cope with this paramount visual impairment problem. In this article, an automatic recognition of the DR stage is proposed based on a new multi-layer architecture of active deep learning (ADL). To develop the ADL system, we used the convolutional neural networks (CNN) model to automatically extract features compare to handcrafted-based features. However, the training of CNN procedure requires an immense size of labeled data that makes it almost difficult in the classification phase. As a result, a label-efficient CNN architecture is presented known as ADL-CNN by using one of the active learning methods known as an expected gradient length (EGL). This ADL-CNN model can be seen as a two-stage process. At first, the proposed ADL-CNN system selects both the most informative patches and images by using some ground truth labels of training samples to learn the simple to complex retinal features. Next, it provides useful masks for prognostication to assist clinical specialists for the important eye sample annotation and segment regions-of-interest within the retinograph image to grade five severity-levels of diabetic retinopathy. To test and evaluate the performance of ADL-CNN model, the EyePACS benchmark is utilized and compared with state-of-the-art methods. The statistical metrics are used such as sensitivity (SE), specificity (SP), F-measure and classification accuracy (ACC) to measure the effectiveness of ADL-CNN system. On 54,000 retinograph images, the ADL-CNN model achieved an average SE of 92.20%, SP of 95.10%, F-measure of 93% and ACC of 98%. Hence, the new ADL-CNN architecture is outperformed for detecting DR-related lesions and recognizing the five levels of severity of DR on a wide range of fundus images.
Similar content being viewed by others
References
Abbas Q, Fondon I, Sarmiento A, Jiménez S, Alemany P (2017) Automatic recognition of severity level for diagnosis of diabetic retinopathy using deep visual features. Med Biol Eng Comput 55(11):1959–1974. https://doi.org/10.1007/s11517-017-1638-6
Abbas Q, Ibrahim MEA, Jaffar MA (2018) Video scene analysis: an overview and challenges on deep learning algorithms. Multimed Tools Appl 77(16):20415–20453. https://doi.org/10.1007/s11042-017-5438-7
Abràmoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Invest Ophthalmol Vis Sci 57(13):5200–5206. https://doi.org/10.1167/iovs.16-19964
Acharya R, Chua CK, Ng EYK, Yu W, Chee C (2008) Application of higher order spectra for the identification of diabetes retinopathy stages. J Med Syst 32(6):481–488. https://doi.org/10.1007/s10916-008-9154-8
Acharya UR, Lim CM, Ng EYK, Chee C, Tamura T (2009) Computer-based detection of diabetes retinopathy stages using digital fundus images. Proceedings of the institution of mechanical engineers, part H: J Eng Med 223, no. 5 : 545–553. https://doi.org/10.1243/09544119jeim486
Adarsh P, Jeyakumari D (2013) Multiclass SVM-based automated diagnosis of diabetic retinopathy. In 2013 International Conference on Communication and Signal Processing, pp. 206–210. IEEE. https://doi.org/10.1109/iccsp.2013.6577044
Akilesh B, Marwah T, Balasubramanian VN, Rajamani K (2017) On the relevance of very deep networks for diabetic retinopathy diagnostics. In Applications of Cognitive Computing Systems and IBM Watson, pp. 47–54. Springer, Singapore. https://doi.org/10.1007/978-981-10-6418-0_6
Alban M, Gilligan T (2016) Automated detection of diabetic retinopathy using fluorescein angiography photographs. Rep Standford Educ
Arunkumar R, Karthigaikumar P (2017) Multi-retinal disease classification by reduced deep learning features. Neural Comput & Applic 28(2):329–334. https://doi.org/10.1007/s00521-015-2059-9
Bello-Cerezo R, Bianconi F, Di Maria F, Napoletano P, Smeraldi F (2019) Comparative Evaluation of Handcrafted Image Descriptors vs Off-the-Shelf CNN-Based Features for Colour Texture Classification under Ideal and Realistic Conditions. Appl Sci 9(4):738. https://doi.org/10.3390/app9040738
Bhaskaranand M, Ramachandra C, Bhat S, Cuadros J, Nittala MG, Sadda SV, Solanki K (2016) Automated diabetic retinopathy screening and monitoring using retinal fundus image analysis. J Diabetes Sci Technol 10(2):254–261. https://doi.org/10.1177/1932296816628546
Chandrakumar T, Kathirvel R (2016) Classifying diabetic retinopathy using deep learning architecture. Int J Eng Res Technol 5(6):19–24. https://doi.org/10.17577/ijertv5is060055
Choi JY, Yoo TK, Seo JG, Kwak J, Um TT, Rim TH (2017) Multi-categorical deep learning neural network to classify retinal images: A pilot study employing small database. PloS one 12(11):e0187336. https://doi.org/10.1371/journal.pone.0187336
Christine N (2015) Your diabetic patients: look them in the eyes. Which ones will lose their sight?. https://www.eyepacs.com/diabeticretinopathy/
Chudzik P, Majumdar S, Calivá F, Al-Diri B, Hunter A (2018) Microaneurysm detection using fully convolutional neural networks. Comput Methods Prog Biomed 158:185–192. https://doi.org/10.1016/j.cmpb.2018.02.016
Colas E, Besse A, Orgogozo A, Schmauch B, Meric N, Besse E (2016) Deep learning approach for diabetic retinopathy screening. Acta Ophthalmol 94. https://doi.org/10.1111/j.1755-3768.2016.0635
Costa P, Campilho A (2017) Convolutional bag of words for diabetic retinopathy detection from eye fundus images. IPSJ Trans Comput Vis Appl 9(1):10. https://doi.org/10.1186/s41074-017-0023-6
Dai L, Fang R, Li H, Hou X, Sheng B, Wu Q, Jia W (2018) Clinical report guided retinal microaneurysm detection with multi-sieving deep learning. IEEE Trans Med Imaging 37(5):1149–1161. https://doi.org/10.1109/tmi.2018.2794988
Decencière E, Cazuguel G, Zhang X, Thibault G, Klein JC, Meyer F, Marcotegui B, Quellec G, Lamard M, Danno R et al (2013) Teleophta: machine learning and image processing methods for teleophthalmology. IRBM 34(2):196–203. https://doi.org/10.1016/j.irbm.2013.01.010
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pp. 248–255. Ieee. https://doi.org/10.1109/cvprw.2009.5206848
Freytag A, Rodner E, Denzler J (2014) Selecting influential examples: Active learning with expected model output changes. In European Conference on Computer Vision, pp. 562–577. Springer, Cham. https://doi.org/10.1007/978-3-319-10593-2_37
Galveia JN, Travassos A, Quadros FA, da Silva Cruz LA (2018) Computer aided diagnosis in ophthalmology: Deep learning applications. In Classification in BioApps, pp. 263–293. Springer, Cham. https://doi.org/10.1007/978-3-319-65981-7_10
García G, Gallardo J, Mauricio A, López J, Del Carpio C (2017) Detection of diabetic retinopathy based on a convolutional neural network using retinal fundus images. In International Conference on Artificial Neural Networks, pp. 635–642. Springer, Cham. https://doi.org/10.1007/978-3-319-68612-7_72
Gardner GG, Keating D, Williamson TH, Elliott AT (1996) Automatic detection of diabetic retinopathy using an artificial neural network: a screening tool. Br J Ophthalmol 80(11):940–944. https://doi.org/10.1136/bjo.80.11.940
Gargeya R, Leng T (2017) Automated identification of diabetic retinopathy using deep learning. Ophthalmology 124(7):962–969. https://doi.org/10.1016/j.ophtha.2017.02.008
Gondal WM, Köhler JM, Grzeszick R, Fink GA, Hirsch M (2017) Weakly-supervised localization of diabetic retinopathy lesions in retinal fundus images. In 2017 IEEE International Conference on Image Processing (ICIP), pp. 2069-2073. IEEE. https://doi.org/10.1109/icip.2017.8296646
Greenspan H, Van Ginneken B, Ronald M (2016) Summers Guest editorial deep learning in medical imaging: Overview and future promise of an exciting new technique. IEEE Trans Med Imaging 35(5):1153–1159. https://doi.org/10.1109/tmi.2016.2553401
Gu J, Wang Z, Kuen J, Ma L, Shahroudy A, Shuai B, Liu T et al (2018) Recent advances in convolutional neural networks. Pattern Recogn 77:354–377. https://doi.org/10.1016/j.patcog.2017.10.013
Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22):2402–2410. https://doi.org/10.1001/jama.2016.17216
Guo S, Li T, Kang H, Li N, Zhang Y, Wang K (2019) L-Seg: An end-to-end unified framework for multi-lesion segmentation of fundus images. Neurocomput 349:52–63. https://doi.org/10.1016/j.neucom.2019.04.019
Guo Y, Yu L, Oerlemans A, Lao S, Wu S, Michael S (2016) Lew Deep learning for visual understanding: A review. Neurocomput 187:27–48. https://doi.org/10.1016/j.neucom.2015.09.116
Haloi M (2015) Improved microaneurysm detection using deep neural networks. arXiv preprint arXiv:1505.04424
Haneda S, Yamashita H (2010) International clinical diabetic retinopathy disease severity scale. Nihon rinsho Jpn J Clin Med 68:228
He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In Proceedings of the IEEE international conference on computer vision, pp. 1026–1034. https://doi.org/10.1109/iccv.2015.123
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778. https://doi.org/10.1109/CVPR.2016.90
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 7132–7141. https://doi.org/10.1109/CVPR.2018.00745
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4700–4708. https://doi.org/10.1109/CVPR.2017.243
Ishtiaq U, Kareem SA, Abdullah ERMF, Mujtaba G, Jahangir R, Ghafoor HY (2019) Diabetic retinopathy detection through artificial intelligent techniques: a review and open issues. Multimed Tools Appl 79:1–44. https://doi.org/10.1007/s11042-018-7044-8
Jia Y, Shelhamer E, Donahue J, Karayev S, Long J, Girshick R, Guadarrama S, Darrell T (2014) Caffe: Convolutional architecture for fast feature embedding. In Proceedings of the 22nd ACM international conference on Multimedia, pp. 675–678. ACM. https://doi.org/10.1145/2647868.2654889
Kermany DS, Goldbaum M, Cai W, Valentim CCS, Liang H, Baxter SL, McKeown A et al (2018) Identifying medical diagnoses and treatable diseases by image-based deep learning. Cell 172(5):1122–1131. https://doi.org/10.1016/j.cell.2018.02.010
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In Advances in neural information processing systems, pp. 1097–1105. https://doi.org/10.1145/3065386
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324. https://doi.org/10.1109/5.726791
Li T, Gao Y, Wang K, Guo S, Liu H, Kang H (2019) Diagnostic Assessment of Deep Learning Algorithms for Diabetic Retinopathy Screening. Inf Sci. https://doi.org/10.1016/j.ins.2019.06.011
Li K, He F, Yu H, Chen X (2019) A parallel and robust object tracking approach synthesizing adaptive Bayesian learning and improved incremental subspace learning. Front Comp Sci 13(5):1116–1135. https://doi.org/10.1007/s11704-018-6442-4
Li B, Helen KL (2013) Automated analysis of diabetic retinopathy images: principles, recent developments, and emerging trends. Curr Diab Rep 13(4):453–459. https://doi.org/10.1007/s11892-013-0393-9
Li Z, Keel S, Liu C, He Y, Meng W, Scheetz J, Lee PY et al (2018) An automated grading system for detection of vision-threatening referable diabetic retinopathy on the basis of color fundus photographs. Diab care 41(12):2509–2516. https://doi.org/10.2337/dc18-0147
Lin J, Yu L, Weng Q, Zheng X (2019) Retinal image quality assessment for diabetic retinopathy screening: a survey. Multimed Tools Appl 79:1–27. https://doi.org/10.1007/s11042-019-07751-6
Liu L, Chen J, Fieguth P, Zhao G, Chellappa R, Pietikäinen M (2019) From BoW to CNN: two decades of texture representation for texture classification. Int J Comput Vis 127(1):74–109. https://doi.org/10.1007/s11263-018-1125-z
Mansour RF (2018) Deep-learning-based automatic computer-aided diagnosis system for diabetic retinopathy. Biomed Eng Lett 8(1):41–57. https://doi.org/10.1007/s13534-017-0047-y
Mookiah MRK, Acharya UR, Chua CK, Lim CM, Ng EYK, Laude A (2013) Computer-aided diagnosis of diabetic retinopathy: A review. Comput Biol Med 43(12):2136–2155. https://doi.org/10.1016/j.compbiomed.2013.10.007
Nanni L, Ghidoni S, Brahnam S (2018) Ensemble of convolutional neural networks for bioimage classification. App Comput Inf. https://doi.org/10.1016/j.aci.2018.06.002
Napoletano P (2017) Handcrafted vs learned descriptors for color texture classification. In International Workshop on Computational Color Imaging, pp. 259–271. Springer, Cham. https://doi.org/10.1007/978-3-319-56010-6_22
Nayak J, Bhat PS, Acharya R, Lim CM, Kagathi M (2008) Automated identification of diabetic retinopathy stages using digital fundus images. J Med Syst 32(2):107–115. https://doi.org/10.1007/s10916-007-9113-9
Orlando JI, Prokofyeva E, del Fresno M, Blaschko MB (2018) An ensemble deep learning based approach for red lesion detection in fundus images. Comput Methods Prog Biomed 153:115–127. https://doi.org/10.1016/j.cmpb.2017.10.017
Otálora S, Perdomo O, González F, Müller H (2017) Training deep convolutional neural networks with active learning for exudate classification in eye fundus images. In Intravascular Imaging and Computer Assisted Stenting, and Large-Scale Annotation of Biomedical Data and Expert Label Synthesis, pp. 146–154. Springer, Cham. https://doi.org/10.1007/978-3-319-67534-3_16
Pan Y, He F, Haiping Y (2020) Learning social representations with deep autoencoder for recommender system. World Wide Web 23:1–21. https://doi.org/10.1007/s11280-020-00793-z
Pang H, Luo C, Wang C (2018) Improvement of the application of diabetic retinopathy detection model. Wirel Pers Commun 103(1):611–624. https://doi.org/10.1007/s11277-018-5465-3
Perdomo O, Otalora S, Rodríguez F, Arevalo J, González FA (2016) A Novel Machine Learning Model Based on Exudate Localization to Detect Diabetic Macular Edema. In: Chen X, Garvin MK, Liu J, Trucco E, Xu Y editors. Proceedings of the Ophthalmic Medical Image Analysis Third International Workshop, OMIA 2016, Held in Conjunction with MICCAI 2016, Athens, Greece, October 21. 137–144. https://doi.org/10.17077/omia.1057
Pratt H, Coenen F, Broadbent DM, Harding SP, Zheng Y (2016) Convolutional neural networks for diabetic retinopathy. International Conference on Medical Imaging Understanding and Analysis, 6-8 July, Loughborough, UK. Procedia Comput Sci 90:200–205. https://doi.org/10.1016/j.procs.2016.07.014
Prentašić P, Lončarić S (2015) Detection of exudates in fundus photographs using convolutional neural networks. In 2015 9th International Symposium on Image and Signal Processing and Analysis (ISPA), pp. 188–192. IEEE. https://doi.org/10.1109/ispa.2015.7306056
Quellec G, Charrière K, Boudi Y, Cochener B, Lamard M (2017) Deep image mining for diabetic retinopathy screening. Med Image Anal 39:178–193. https://doi.org/10.1016/j.media.2017.04.012
Qureshi I (2015) Glaucoma detection in retinal images using image processing techniques: a survey. Int J Adv Netw Appl 7(2):2705
Qureshi I, Khan MA, Sharif M, Saba T, Ma J (2020) Detection of glaucoma based on cup-to-disc ratio using fundus images. Int J Intell Syst Technol Appl 19(1):1–16. https://doi.org/10.1504/IJISTA.2020.105172
Qureshi I, Ma J, Abbas Q (2019) Recent development on detection methods for the diagnosis of diabetic retinopathy. Symmetry 11(6):749. https://doi.org/10.3390/sym11060749
Qureshi I, Ma J, Shaheed K (2019) A hybrid proposed fundus image enhancement framework for diabetic retinopathy. Algorithms 12(1):1–14. https://doi.org/10.3390/a12010014
Qureshi I, Sharif M, Yasmin M, Raza M, Javed MY (2016) Computer aided systems for diabetic retinopathy detection using digital fundus images: A survey. Curr Med Imaging Rev 12(4):234–241. https://doi.org/10.2174/1573405611666150929234644
Rahim SS, Palade V, Shuttleworth J, Jayne C (2016) Automatic screening and classification of diabetic retinopathy and maculopathy using fuzzy image processing. Brain Inform 3(4):249–267. https://doi.org/10.1007/s40708-016-0045-3
Raju M, Pagidimarri V, Barreto R, Kadam A, Kasivajjala V, Aswath A (2017) Development of a Deep Learning Algorithm for Automatic Diagnosis of Diabetic Retinopathy. In MedInfo, pp. 559–563. doi:https://doi.org/10.3233/978-1-61499-830-3-559
Sadek I, Elawady M, Shabayek AER (2017) Automatic classification of bright retinal lesions via deep network features. arXiv preprint arXiv:1707.02022
Saha SK, Fernando B, Cuadros J, Xiao D, Kanagasingam Y (2018) Automated quality assessment of colour fundus images for diabetic retinopathy screening in telemedicine. J Digit Imaging 31(6):869–878. https://doi.org/10.1007/s10278-018-0084-9
Sánchez CI, Niemeijer M, Abràmoff MD, van Ginneken B (2010) Active learning for an efficient training strategy of computer-aided diagnosis systems: application to diabetic retinopathy screening. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 603–610. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15711-0_75
Sankar M, Batri K, Parvathi R (2016) Earliest diabetic retinopathy classification using deep convolution neural networks. Int J Adv Eng Technol
Settle B Active learning. Synthesis lectures on artificial intelligence and machine learning 6 (2012): pp. 1–114. https://doi.org/10.2200/s00429ed1v01y201207aim018
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556
Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Chua KC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:66–76. https://doi.org/10.1016/j.ins.2017.08.050
Ting DSW, Cheung CY-L, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H et al (2017) Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 318(22):2211–2223. https://doi.org/10.1001/jama.2017.18152
Ting DSW, Cheung GCM, Wong TY (2016) Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin Exp Ophthalmol 44(4):260–277. https://doi.org/10.1111/ceo.12696
Wang Z, Yang J (2017) Diabetic retinopathy detection via deep convolutional networks for discriminative localization and visual explanation. arXiv preprint: http://arXiv.org/abs/1703.10757
Wang Z, Yin Y, Shi J, Fang W, Li H, Wang X (2017) Zoom-in-net: Deep mining lesions for diabetic retinopathy detection. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 267–275. Springer, Cham. https://doi.org/10.1007/978-3-319-66179-7_31
Washington RE, Orchard TJ, Arena VC, LaPorte RE, Secrest AM, Tull ES (2014) All-cause mortality in a population-based type 1 diabetes cohort in the US Virgin Islands. Diabetes Res Clin Pract 103(3):504–509. https://doi.org/10.1016/j.diabres.2013.12.014
Wu Y, He F, Zhang D, Li X (2015) Service-oriented feature-based data exchange for cloud-based design and manufacturing. IEEE Trans Serv Comput 11(2):341–353. https://doi.org/10.1109/TSC.2015.2501981
Xu K, Feng D, Mi H (2017) Deep convolutional neural network-based early automated detection of diabetic retinopathy using fundus image. Molecules 22(12):2054. https://doi.org/10.3390/molecules22122054
Yang Y, Li T, Li W, Wu H, Fan W, Zhang W (2017) Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 533–540. Springer, Cham. https://doi.org/10.1007/978-3-319-66179-7_61
Yu F, Seff A, Zhang Y, Song S, Funkhouser T, Xiao J (2015) Lsun: Construction of a large-scale image dataset using deep learning with humans in the loop. arXiv preprint arXiv:1506.03365
Zhang J, He F, Chen Y (2020) A new haze removal approach for sky/river alike scenes based on external and internal clues. Multimed Tools Appl 79(3):2085–2107. https://doi.org/10.1007/s11042-019-08399-y
Zhang W, Zhong J, Yang S, Gao Z, Hu J, Chen Y, Yi Z (2019) Automated identification and grading system of diabetic retinopathy using deep neural networks. Knowl-Based Syst 175:12–25. https://doi.org/10.1016/j.knosys.2019.03.016
Zhou S, Chen Q, Wang X (2014) Active semi-supervised learning method with hybrid deep belief networks. PLoS One 9(9):e107122. https://doi.org/10.1371/journal.pone.0107122
Zhou L, Yu Z, Yang J, Yu Q, Xun X (2017) Deep multiple instance learning for automatic detection of diabetic retinopathy in retinal images. IET Image Process 12(4):563–571. https://doi.org/10.1049/iet-ipr.2017.0636
Acknowledgments
We would like to thank all the anonymous reviewers for their valuable comments. This work is supported by the National Natural Science Foundation of China (NNSFC) under the grant no: (61672322, 61672324) and is a part of a Ph.D. research project conducted in the intelligent and media research center (iLEARN), School of Computer Science and Technology, Shandong University, Qingdao, China.
Funding
This work is supported by the National Natural Science Foundation of China (NNSFC) under grant no: (61672322, 61672324).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors disclosed no potential conflicts of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Qureshi, I., Ma, J. & Abbas, Q. Diabetic retinopathy detection and stage classification in eye fundus images using active deep learning. Multimed Tools Appl 80, 11691–11721 (2021). https://doi.org/10.1007/s11042-020-10238-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-020-10238-4