Abstract
Diabetic Retinopathy is one of the prominent reasons for permanent blindness in working age, long term diabetic patients. With the prevalence in raise of diabetics, majority of the people are endangered to permanent vision loss. The advancements in medical imaging techniques enabled the research community to focus on developing automated and computerized systems for diagnosing retinopathy in early stages. But, it is a very complex challenge due to the presence of high intra-class variations and imbalanced data distribution for higher grades of severity. In recent years, various deep learning based models have been designed for automating the process of retinopathy severity classification. In this research work, we present a fascinating deep learning model with multiple attention stages called Hinge Attention Network (HA-Net). Proposed model consists of a pre-trained VGG16 base to extract initial spatial representation from retinal scan images, spatial attention autoencoder to learn lesion specific latent representations in spatial dimensions and a channel attention based hinge neural network to grab category based discriminative features in channel dimension and classify the severity grade of retinopathy. In addition to spatial and channel attention mechanism, we use Convolutional LSTM layer to prioritize highly important spatial maps before passing to hinge neural network. All these components of HA-Net, enabled it to make generalised and accurate predictions on unseen data. The effectiveness and acceptability of proposed model is proved by validating it using two benchmark datasets, Kaggle APTOS 2019 and ISBI IDRiD. Extensive experimental studies on these datasets reveal that, proposed HA-Net outstrip several existing models by achieving an accuracy of 85.54% on Kaggle APTOS, and an accuracy of 66.41% on IDRiD datasets.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abràmoff MD, Lou Y, Erginay A, Clarida W, Amelon R, Folk JC, Niemeijer M (2016) Improved automated detection of diabetic retinopathy on a publicly available dataset through integration of deep learning. Investigative Ophthalmology & Visual Science 57(13):5206
Al-Antary MT, Arafa Y (2021) Multi-scale attention network for diabetic retinopathy classification. IEEE Access 9:54190–54200
Amin J, Sharif M, Yasmin M (2016) A review on recent developments for detection of diabetic retinopathy. Scientifica
Bhandary SV, Rao KA (2018) Automated screening system for retinal health using bi-dimensional empirical mode decomposition and integrated index. Computers in Biology and Medicine 75:54–62
Bodapati JD, Shaik NS, Naralasetti V (2021) Composite deep neural network with gated-attention mechanism for diabetic retinopathy severity classification. J Ambient Intell Humaniz Comput
Bodapati JD, Shaik NS, Naralasetti V (2021) Deep convolution feature aggregation: an application to diabetic retinopathy severity level prediction. Signal Image Video Process:1–8
Bodapati JD, Shaik NS, Naralasetti V, Mundukur NB (2020) Joint training of two-channel deep neural network for brain tumor classification. Signal Image Video Process:1–8
Bodapati JD, Shareef SN, Naralasetti V, Mundukur NB (2021) Msenet: Multi-modal squeeze-and-excitation network for brain tumor severity prediction. Int J Pattern Recognit Artif Intell:2157005
Bodapati JD, Veeranjaneyulu N, Shareef SN, Hakak S, Bilal M, Maddikunta PKR, Jo O (2020) Blended multi-modal deep convnet features for diabetic retinopathy severity prediction. Electronics 9(6):914
Chen M, Shi X, Zhang Y, Wu D, Guizani M (2017) Deep features learning for medical image analysis with convolutional autoencoder neural network. IEEE Trans Big Data:1–1
Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1251–1258
Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. Ieee, pp 248–255
Dondeti V, Bodapati JD, Shareef SN, Naralasetti V (2020) Deep convolution features in non-linear embedding space for fundus image classification deep convolution features in non-linear embedding space for fundus image classification. Revue d’Intelligence Artificielle 34(3):307–313
Eisenbarth GS (1986) Type i diabetes mellitus. New England Journal of Medicine 314(21):1360–1368
Fukui H, Hirakawa T, Yamashita T, Fujiyoshi H (2019) Attention branch network: Learning of attention mechanism for visual explanation. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR)
Gao Z, Li J, Guo J, Chen Y, Yi Z, Zhong J (2019) Diagnosis of diabetic retinopathy using deep neural networks. IEEE Access 7:3360–3370
Habib M, Welikala R, Hoppe A, Owen C, Rudnicka A, Barman S (2017) Detection of microaneurysms in retinal images using an ensemble classifier. Informatics in Medicine Unlocked 9:44–57
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
International diabetes federation diabetes atlas (2019) https://www.diabetesatlas.org/en/. Accessed: 30-06-2021
Janghorbani M, Jones RB, Allison SP (2000) Incidence of and risk factors for proliferative retinopathy and its association with blindness among diabetes clinic attenders. Ophthalmic Epidemiology 7(4):225–241
Kaggle (2019) Aptos 2019 blindness detection challenge. https://www.kaggle.com/c/aptos2019-blindnes-detection. Accessed: 2019-12-30
Kandel I, Castelli M (2020) Transfer learning with convolutional neural networks for diabetic retinopathy image classification. a review. Applied Sciences 10(6):2021
Kassani SH, Kassani PH, Khazaeinezhad R, Wesolowski MJ, Schneider KA, Deters R (2019) Diabetic retinopathy classification using a modified xception architecture. In: 2019 IEEE International symposium on signal processing and information technology (ISSPIT). IEEE, pp 1–6
Kaur N, Chatterjee S, Acharyya M, Kaur J, Kapoor N, Gupta S (2016) A supervised approach for automated detection of hemorrhages in retinal fundus images. In: 2016 5th International conference on wireless networks and embedded systems (WECON). IEEE, pp 1–5
Li X, Hu X, Yu L, Zhu L, Fu C-W, Heng P-A (2019) Canet: Cross-disease attention network for joint diabetic retinopathy and diabetic macular edema grading. IEEE Transactions on Medical Imaging 39(5):1483–1493
Long S, Huang X, Chen Z, Pardhan S, Zheng D (2019) Automatic detection of hard exudates in color retinal images using dynamic threshold and svm classification: algorithm development and evaluation. BioMed Res Int
Luong M-T, Pham H, Manning CD (2015) Effective approaches to attention-based neural machine translation. In: International conference on learning representations
Martinez-Murcia FJ, Ortiz A, Ramírez J, Górriz JM, Cruz R (2021) Deep residual transfer learning for automatic diagnosis and grading of diabetic retinopathy. Neurocomputing 452:424–434
Mateen M, Wen J, Song S, Huang Z et al (2019) Fundus image classification using vgg-19 architecture with pca and svd. Symmetry 11(1):1
Mohammedhasan M, Uğuz H (2020) A new early stage diabetic retinopathy diagnosis model using deep convolutional neural networks and principal component analysis. Traitement du Signal 37(5):711–722
Nanni L, Ghidoni S, Brahnam S (2017) Handcrafted vs. non-handcrafted features for computer vision classification. Pattern Recognition 71:158–172
Noushin E, Pourreza M, Masoudi K, Ghiasi Shirazi E (2019) Microaneurysm detection in fundus images using a two step convolution neural network. Biomed Eng Online 18(1):67
Pires R, Avila S, Wainer J, Valle E, Abramoff MD, Rocha A (2019) A data-driven approach to referable diabetic retinopathy detection. Artificial Intelligence in Medicine 96:93–106
Porwal P, Pachade S, Kamble R, Kokare M, Deshmukh G, Sahasrabuddhe V, Meriaudeau F (2018) Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research. Data 3(3):25
Porwal P, Pachade S, Kokare M, Deshmukh G, Son J, Bae W, Liu L, Wang J, Liu X, Gao L et al (2020) Idrid: Diabetic retinopathy-segmentation and grading challenge. Medical Image Analysis 59:101561
Prentašić P, Lončarić S (2016) Detection of exudates in fundus photographs using deep neural networks and anatomical landmark detection fusion. Computer Methods and Programs in Biomedicine 137:281–292
Quellec G, Al Hajj H, Lamard M, Conze P-H, Massin P, Cochener B (2021) Explain: Explanatory artificial intelligence for diabetic retinopathy diagnosis. Med Image Anal:102118
Qureshi I, Ma J, Abbas Q (2019) Recent development on detection methods for the diagnosis of diabetic retinopathy. Symmetry 11(6):749
Rahim SS, Palade V, Holzinger A (2020) Image processing and machine learning techniques for diabetic retinopathy detection: A review. Artif Intell Mach Learn Digital Pathology:136–154
Razzak MI, Naz S, Zaib A (2018) Deep learning for medical image processing: Overview, challenges and the future. In: Classification in BioApps. Springer, pp 323–350
Reza AM (2004) Realization of the contrast limited adaptive histogram equalization (clahe) for real-time image enhancement. Journal of VLSI Signal Processing Systems for Signal, Image and Video Technology 38(1):35–44
Shaban M, Ogur Z, Mahmoud A, Switala A, Shalaby A, Abu Khalifeh H, Ghazal M, Fraiwan L, Giridharan G, Sandhu H et al (2020) A convolutional neural network for the screening and staging of diabetic retinopathy. Plos one 15(6):e0233514
Shaik NS, Cherukuri TK (2021) Lesion-aware attention with neural support vector machine for retinopathy diagnosis. Machine Vision and Applications 32(6):1–13
Shaik NS, Cherukuri TK (2021) Multi-level attention network: application to brain tumor classification. Signal Image Video Process:1–8
Shi X, Chen Z, Wang H, Yeung D-Y, Wong W-K, WOO, W-C (2015) Convolutional lstm network: A machine learning approach for precipitation nowcasting. In: Advances in neural information processing systems. vol 28, Curran Associates, Inc., pp 802–810
Sikder N, Masud M, Bairagi AK, Arif ASM, Nahid A-A, Alhumyani HA (2021) Severity classification of diabetic retinopathy using an ensemble learning algorithm through analyzing retinal images. Symmetry 13(4):670
Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:1409.1556
Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. In: Advances in neural information processing systems. vol 28, Curran Associates, Inc., pp 2377–2385
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826
Tsiknakis N, Theodoropoulos D, Manikis G, Ktistakis E, Boutsora O, Berto A, Scarpa F, Scarpa A, Fotiadis DI, Marias K (2021) Deep learning for diabetic retinopathy detection and classification based on fundus images: A review. Comput Biol Med:104599
Van der Maaten L, Hinton G (2008) Visualizing data using t-sne. Journal of machine learning research 9:11
Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhudinov R, Zemel R, Bengio Y (2015) Show, attend and tell: Neural image caption generation with visual attention. In: Proceedings of machine learning research (Lille, France, 09 2015), vol 37, PMLR, pp 2048–2057
Yau JW, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, Chen S-J, Dekker JM, Fletcher A, Grauslund J et al (2012) prevalence and major risk factors of diabetic retinopathy. Diabetes care 35(3):556–564
Zheng Y, He M, Congdon N (2012) The worldwide epidemic of diabetic retinopathy. Indian journal of ophthalmology 60(5):428
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Shaik, N.S., Cherukuri, T.K. Hinge attention network: A joint model for diabetic retinopathy severity grading. Appl Intell 52, 15105–15121 (2022). https://doi.org/10.1007/s10489-021-03043-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-021-03043-5