Skip to main content
SpringerLink
Log in

Automated detection of retinopathy of prematurity by deep attention network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Retinopathy of prematurity (ROP) is a retinal vascular proliferative disease principally observed in infants born prematurely with low birth weight. ROP is the leading cause of childhood blindness. Early screening and timely treatment are crucial in preventing ROP blindness. Previous ROP diagnosis lacks clear understanding of the underlying factors and properties that supports the final decision. For this reason, a deep convolutional neural network (DCNN) is developed for automated ROP detection using wide-angle retinal images. Specifically, we first choose ResNet50 as our base architecture and improve the ResNet by adding a channel and a spatial attention module. Then, we utilize a class-discriminative localization technique (i.e., gradient-weighted class activation mapping (Grad-CAM)) to visualize the trained models and realize pathological structure localization. The efficacy of the proposed network is evaluated on two test datasets. Our method obtains a sensitivity of 94.84   % and a specificity of 99.49   % on test set 1 while a sensitivity of 98.03   % and a specificity of 94.55   % on test set 2. Also, the model successfully detects the pathological structures of ROP (e.g., demarcation lines or ridges) in the retina images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RP et al (2018) Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol 136(7):803–810

  2. Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, Bressler NM (2017) Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol 135(11):1170–1176

    Article  Google Scholar 

  3. Chen Y, Feng J, Gilbert C, Yin H, Liang J, Li X (2015) Time at treatment of severe retinopathy of prematurity in China: recommendations for guidelines in more mature infants. PLoS One 10(2):e0116669

    Article  Google Scholar 

  4. Chen L-C, Papandreou G, Schroff F, Adam H (2017) Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587

  5. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. Proc IEEE Europ Conf Comput Vis, pp 801–818

    Google Scholar 

  6. Chiang MF, Jiang L, Gelman R, Du YE, Flynn JT (2007) Interexpert agreement of plus disease diagnosis in retinopathy of prematurity. Arch Ophthalmol 125(7):875–880

    Article  Google Scholar 

  7. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1251–1258

  8. Diaz M, Ferrer MA, Impedovo D, Pirlo G, Vessio G (2019) Dynamically enhanced static handwriting representation for Parkinson’s disease detection. Pattern Recogn Lett 128:204–210

    Article  Google Scholar 

  9. Esteva A, Kuprel B, Novoa RA, Ko J, Swetter SM, Blau HM et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542(7639):115–118

    Article  Google Scholar 

  10. Early Treatment For Retinopathy Of Prematurity Cooperative Group (2003) Revised indications for the treatment of retinopathy of prematurity: results of the early treatment for retinopathy of prematurity randomized trial. Arch Ophthalmol 121:1684

    Article  Google Scholar 

  11. Fu J, Liu J, Tian H, Fang Z, Lu H (2018) Dual attention network for scene segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 3146–3154

  12. Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 580–587

  13. Good WV, Hardy RJ, Dobson V, Palmer EA, Phelps DL, Tung B et al (2010) Final visual acuity results in the early treatment for retinopathy of prematurity study. Arch Ophthalmol 128(6):663–671

    Article  Google Scholar 

  14. Gschließer A, Stifter E, Neumayer T, Moser E, Papp A, Pircher N et al (2015) Inter-expert and intra-expert agreement on the diagnosis and treatment of retinopathy of prematurity. Am J Ophthalmol 160(3):553–560.e3

    Article  Google Scholar 

  15. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316(22):2402–2410

    Article  Google Scholar 

  16. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 770–778

    Google Scholar 

  17. Hellström A, Smith LEH, Dammann O (2013) Retinopathy of prematurity. Lancet 382(9902):1445–1457

    Article  Google Scholar 

  18. Hu J, Chen Y, Zhong J, Ju R, Yi Z (2018) Automated analysis for retinopathy of prematurity by deep neural networks. IEEE Trans Med Imaging 38(1):269–279

    Article  Google Scholar 

  19. Hutchinson AK, Melia M, Yang MB, VanderVeen DK, Wilson LB, Lambert SR (2016) Clinical models and algorithms for the prediction of retinopathy of prematurity: a report by the American Academy of Ophthalmology. Ophthalmology 123(4):804–816

    Article  Google Scholar 

  20. I. C. f. t. C. o. R. o. Prematurity (1984) An international classification of retinopathy of prematurity. Arch Ophthalmol 102:1130–1134

    Article  Google Scholar 

  21. I. C. f. C. o. L. S. ROP (1987) An international classification of retinopathy of prematurity: II. The classification of retinal detachment. Arch Ophthalmol 105(7):906–912

    Article  Google Scholar 

  22. I. C. f. t. C. o. R. o. Prematurity (2005) The international classification of retinopathy of prematurity revisited. Arch Ophthalmol 123(7):991–999

    Article  Google Scholar 

  23. Jia X, Shen L, Zhou X, Yu S (2016) Deep convolutional neural network based HEp-2 cell classification. In: (2016) 23rd International Conference on Pattern Recognition (ICPR), pp 77–80

    Chapter  Google Scholar 

  24. Khan MA, Kadry S, Alhaisoni M, Nam Y, Zhang Y, Rajinikanth V et al (2020) Computer-aided gastrointestinal diseases analysis from wireless capsule endoscopy: A framework of best features selection. IEEE Access 8:132850–132859

    Article  Google Scholar 

  25. Khan MA, Arshad H, Nisar W, Javed MY, Sharif M (2021) An integrated design of fuzzy C-means and NCA-based multi-properties feature reduction for brain tumor recognition. In: Signal and Image Processing Techniques for the Development of Intelligent Healthcare Systems. Springer, Berlin, pp 1–28

    Google Scholar 

  26. Kim SJ, Port AD, Swan R, Campbell JP, Chan RP, Chiang MF (2018) Retinopathy of prematurity: a review of risk factors and their clinical significance. Surv Ophthalmol 63(5):618–637

    Article  Google Scholar 

  27. Kimyon S, Mete A (2018) Comparison of bevacizumab and ranibizumab in the treatment of type 1 retinopathy of prematurity affecting zone 1. Ophthalmologica 240(2):1–7

    Article  Google Scholar 

  28. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inf Process Syst:1097–1105

  29. Liaqat A, Khan M, Sharif M, Mittal M, Saba T, Manic K et al (2020) Gastric tract infections detection and classification from wireless capsule endoscopy using computer vision techniques: a review. Curr Med Imaging 16(10):1229–1242

    Article  Google Scholar 

  30. Lin T-Y, Goyal P, Girshick R, He K, Dollár P (2017) Focal loss for dense object detection. In: Proc IEEE Int Conf Comput Vis, pp 2980–2988

    Google Scholar 

  31. Liu M, Cheng D, Yan W (2018) Classification of Alzheimer’s disease by combination of convolutional and recurrent neural networks using FDG-PET images. Front Neuroinform 12:35

    Article  Google Scholar 

  32. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 3431–3440

  33. Long E, Lin H, Liu Z, Wu X, Wang L, Jiang J et al (2017) An artificial intelligence platform for the multihospital collaborative management of congenital cataracts. Nat Biomed Eng 1(2):0024

    Article  Google Scholar 

  34. Lvd M, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579–2605

    MATH  Google Scholar 

  35. Majid A, Khan MA, Yasmin M, Rehman A, Yousafzai A, Tariq U (2020) Classification of stomach infections: A paradigm of convolutional neural network along with classical features fusion and selection. Microsc Res Tech 83(5):562–576

    Article  Google Scholar 

  36. Masumoto H, Tabuchi H, Nakakura S, Ishitobi N, Miki M, Enno H (2018) Deep-learning classifier with an ultrawide-field scanning laser ophthalmoscope detects glaucoma visual field severity. J Glaucoma 27(7):647–652

    Article  Google Scholar 

  37. Paszke A, Gross S, Chintala S, Chanan G, Yang E, Z. DeVito, et al (2017) Automatic differentiation in pytorch

  38. Quinn GE, Gilbert C, Darlow BA, Zin A (2010) Retinopathy of prematurity: an epidemic in the making. Chin Med J (Engl) 123(20):2929–2937

    Google Scholar 

  39. Rao J, Fan D, Wu S, Lin D, Zhang H, Ye S et al (2018) Trend and risk factors of low birth weight and macrosomia in south China, 2005–2017: a retrospective observational study. Sci Rep 8(1):3393

    Article  Google Scholar 

  40. Roy AG, Navab N, Wachinger C (2019) Recalibrating fully convolutional networks with spatial and channel “Squeeze and Excitation” blocks. IEEE Trans Med Imaging 38(2):540–549

    Article  Google Scholar 

  41. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 618–626

  42. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv:preprint arXiv:1409.1556

  43. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D et al (2015) Going deeper with convolutions. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1–9

  44. Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. Proc AAAI Conf Artif Intell

  45. Ting DS, Wu W-C, Toth C (2018) Deep learning for retinopathy of prematurity screening. Br J Ophthalmol

  46. Wang J, Ju R, Chen Y, Zhang L, Hu J, Wu Y et al (2018) Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine 35:361–368

    Article  Google Scholar 

  47. Woo S, Park J, Lee J-Y, So Kweon I (2018) Cbam: Convolutional block attention module. In: Proc IEEE Europ Conf Comput Vis, pp 3–19

  48. Wu C, Petersen RA, Vanderveen DK (2006) RetCam imaging for retinopathy of prematurity screening. J AAPOS 10(2):107–111

    Article  Google Scholar 

  49. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1492–1500

    Google Scholar 

  50. Zagoruyko S, Komodakis N (2016) Paying more attention to attention: Improving the performance of convolutional neural networks via attention transfer. arXiv:preprint arXiv:1612.03928

  51. Zahoor S, Lali IU, Khan M, Javed K, Mehmood W (2020) Breast cancer detection and classification using traditional computer vision techniques: a comprehensive review. Curr Med Imaging 16(10):1187–1200

    Article  Google Scholar 

  52. Zeiler MD, Fergus R (2014) Visualizing and understanding convolutional networks. In: Proc IEEE Europ Conf Comput Vis, pp 818–833

    Google Scholar 

  53. Zhang Y, Wang L, Wu Z, Zeng J, Chen Y, Tian R et al (2018) Development of an automated screening system for retinopathy of prematurity using a deep neural network for wide-angle retinal images. IEEE Access 7:10232–10241

    Article  Google Scholar 

  54. Zhang H, Goodfellow I, Metaxas D, Odena A (2018) Self-attention generative adversarial networks. In: International conference on machine learning, pp 7354–7363

    Google Scholar 

  55. Zheng X, Chen W, You Y, Jiang Y, Li M, Zhang T (2020) Ensemble deep learning for automated visual classification using EEG signals. Pattern Recognit 102:107147

    Article  Google Scholar 

  56. Zheng X, Chen W, You Y, Jiang Y, Li M, Zhang T (2020) Ensemble deep learning for automated visual classification using EEG signals. Pattern Recognit 102:107147

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported partly by Shenzhen Key Medical Discipline Construction Fund (No. SZXK038), Shenzhen Fund for Guangdong Provincial High-level Clinical Key Specialties (No.SZGSP014), Shenzhen-Hong Kong Co-financing Project (No.SGDX20190920110403741), and Guangdong Basic and Applied Basic Research Foundation (No. 2019A1515111205).

Author information

Author notes

  1. Baiying Lei and Xianlu Zeng contributed equally to this work.

Authors and Affiliations

  1. School of Biomedical Engineering, Health Science Center, National-Regional Key Technology Engineering Laboratory for Medical Ultrasound, Guangdong Key Laboratory for Biomedical Measurements and Ultrasound Imaging, Shenzhen University, Nanhai Ave 3688, Shenzhen, Guangdong, 518060, China

    Baiying Lei, Shan Huang, Rugang Zhang, Guozhen Chen & Tianfu Wang

  2. Shenzhen Key Ophthalmic Laboratory, Health Science Center, Shenzhen Eye Hospital, Shenzhen University, The Second Affiliated Hospital of Jinan University, Shenzhen, China

    Xianlu Zeng, Jinfeng Zhao, Jiantao Wang & Guoming Zhang

Authors
  1. Baiying Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xianlu Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shan Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rugang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Guozhen Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jinfeng Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Tianfu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jiantao Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Guoming Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jiantao Wang or Guoming Zhang.

Ethics declarations

Competing interests

We wish to confirm that there have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, B., Zeng, X., Huang, S. et al. Automated detection of retinopathy of prematurity by deep attention network. Multimed Tools Appl 80, 36341–36360 (2021). https://doi.org/10.1007/s11042-021-11208-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11208-0

Keywords

Search

Navigation