Skip to main content
SpringerLink
Log in

Deep dive in retinal fundus image segmentation using deep learning for retinopathy of prematurity

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Segmentation of retinal structures, namely optic disc, vessel, demarcation line, and ridge, is essential for describing the characteristics of Retinopathy of Prematurity (ROP). Computerized systems are being developed for automatic segmentation in fundus images to assist the medical experts and bring consistency in the diagnosis. There are multiple challenges in the segmentation task of premature infants’ fundus images. The annotation and ground truth preparation required for the segmentation is complex, challenging, and expensive. Further, ROP datasets are not available publicly, and hence carrying out such a task needs a primary dataset and significant assistance from the domain expert. To address this gap, two primary datasets named HVDROPDB-BV and HVDROPDB-RIDGE were developed. The datasets consist of images captured by two different imaging systems, having different sizes, resolutions, and illumination. This made the trained models generic and robust to data variability and heterogeneity. We propose the modified U-Net architectures by incorporating squeeze and excitation (SE) blocks and attention gates (AG) to segment the demarcation line/ridge and vessel from these datasets. These modifications were tested and validated by ROP experts. The performance of all the three networks (U-Net, AG U-Net, and SE U-Net) was promising, with a variation of 1 to 6% in the dice coefficient for the HVDROPDB datasets. The area under the curve (AUC) obtained for all three networks was above 0.94, indicating them as excellent models. AG U-Net outperformed the other two networks, with a sensitivity of 96% and specificity of 89% for stage detection via new test images.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17

Similar content being viewed by others

References

  1. Agrawal R, Agrawal M, Kulkarni S, Kotecha K, Walambe R (2021) Quantitative analysis of research on artificial intelligence in retinopathy of prematurity. Libr Philos Pract:1–29

  2. Agrawal R, Kulkarni S, Walambe R, Kotecha K (2021) Assistive framework for automatic detection of all the zones in retinopathy of prematurity using deep learning. J Digit Imaging 34:1–16. https://doi.org/10.1007/s10278-021-00477-8

    Article  Google Scholar 

  3. Ahmed, M. (2020). Medical image segmentation using attention-based deep neural networks.

  4. Brown JM, Campbell JP, Beers A, Chang K, Ostmo S, Chan RP, Chiang MF (2018) Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol 136(7):803–810

    Article  Google Scholar 

  5. Chen, C., Chuah, J. H., Raza, A., & Wang, Y. (2021). Retinal vessel segmentation using deep learning: a review. IEEE Access, Retinal Vessel Segmentation Using Deep Learning: A Review.

  6. Chen C, Qin C, Qiu H, Tarroni G, Duan J, Bai W, Rueckert D (2020) Deep learning for cardiac image segmentation: a review. Front Cardiovasc Med 7:25

    Article  Google Scholar 

  7. Chen, G., Zhao, J., Zhang, R., Wang, T., Zhang, G., & Lei, B. (2019, October). Automated stage analysis of retinopathy of prematurity using joint segmentation and multi-instance learning. In international workshop on ophthalmic medical image analysis (pp. 173-181). Springer, Cham.

  8. Chen, L., Zhang, H., Xiao, J., Nie, L., Shao, J., Liu, W., & Chua, T. S. (2017). Sca-cnn: spatial and channel-wise attention in convolutional networks for image captioning. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 5659-5667), arXiv:1611.05594

  9. Ding, A., Chen, Q., Cao, Y., & Liu, B. (2020). Retinopathy of prematurity stage diagnosis using object segmentation and convolutional neural networks arXiv preprint arXiv: 2004.01582.

  10. Du G, Cao X, Liang J, Chen X, Zhan Y (2020) Medical image segmentation based on u-net: a review. J Imaging Sci Technol 64(2):20508–20501

    Article  Google Scholar 

  11. Gojić, G., Petrović, V., Turović, R., Dragan, D., Oros, A., Gajić, D., & Horvat, N. (2020). Deep learning methods for retinal blood vessel segmentation: evaluation on images with retinopathy of prematurity. In2020 IEEE 18th international symposium on intelligent systems and informatics (SISY)(pp. 131-136). IEEE.

  12. Guan, Q., Huang, Y., Zhong, Z., Zheng, Z., Zheng, L., Yang, Y., 2018. Diagnose like a radiologist: attention guided convolutional neural network for thorax disease classification. arXiv:1801.09927.

  13. Harouni, M., &Baghmaleki, H. Y. (2018). Color image segmentation metrics. Encyclopedia of image processing, 95, arXiv:2010.09907.

  14. Hu J, Chen Y, Zhong J, Ju R, Yi Z (2018) Automated analysis for retinopathy of prematurity by deep neural networks. IEEE Trans Med Imaging 38(1):269–279

    Article  Google Scholar 

  15. Hu, J., Shen, L., & Sun, G. (2018). Squeeze-and-excitation networks. InProceedings of the IEEE conference on computer vision and pattern recognition (pp. 7132-7141), https://doi.org/10.1109/CVPR.2018.00745.

  16. Huang YP, Vadloori S, Chu HC, Kang EYC, Wu WC, Kusaka S, Fukushima Y (2020) Deep learning models for automated diagnosis of retinopathy of prematurity in preterm infants. Electronics 9(9):1444

    Article  Google Scholar 

  17. Imran A, Li J, Pei Y, Yang JJ, Wang Q (2019) Comparative analysis of vessel segmentation techniques in retinal images. IEEE Access 7:114862–114887

    Article  Google Scholar 

  18. International Committee for the Classification of Retinopathy of Prematurity (2005) The international classification of Retinopathy of prematurity revisited. Arch Ophthalmol (Chicago, Ill.: 1960) 123(7):991. https://doi.org/10.1001/archopht.123.7.991

    Article  Google Scholar 

  19. Jetley, S., Lord, N. A., Lee, N., & Torr, P. H. (2018). Learn to pay attention. arXiv preprint arXiv:1804.02391.

  20. Khanh TLB, Dao DP, Ho NH, Yang HJ, Baek ET, Lee G, Yoo SB (2020) Enhancing U-net with spatial-channel attention gate for abnormal tissue segmentation in medical imaging. Appl Sci 10(17):5729

    Article  Google Scholar 

  21. Lei B, Zeng X, Huang S, Zhang R, Chen G, Zhao J, Wang T, Wang J, Zhang G (2021) Automated detection of retinopathy of prematurity by deep attention network. Multimed Tools Appl 80:36341–36360. https://doi.org/10.1007/s11042-021-11208-0

    Article  Google Scholar 

  22. Milletari, F. (2018). Hough voting strategies for segmentation, detection and tracking (Doctoral dissertation, Technische Universität München).

  23. Mittal K, Rajam VMA (2020) Computerized retinal image analysis-a survey. Multimed Tools Appl 79:22389–22421

    Article  Google Scholar 

  24. Molinari A, Weaver D, Jalali S (2017) Classifying retinopathy of prematurity. Commu Eye Health 30(99):55–56

    Google Scholar 

  25. Mulay S, Ram K, Sivaprakasam M, Vinekar A (2019) Early detection of retinopathy of prematurity stage using deep learning approach. In medical imaging 2019: computer-aided diagnosis (Vol. 10950, p. 109502Z). Int Soc Optics Photonics. https://doi.org/10.1117/12.2512719

  26. Murki S, Kadam S (2018) Role of a neonatal team including nurses in prevention of ROP. Comm Eye Health 31(101):S11–S15

    Google Scholar 

  27. Oktay, O., Schlemper, J., Folgoc, L. L., Lee, M., Heinrich, M., Misawa, K., &Glocker, B. (2018). Attention u-net: learning where to look for the pancreas. arXiv preprint arXiv:1804.03999.

  28. Razzak, M. I., Naz, S., &Zaib, A. (2018). Deep learning for medical image processing: overview, challenges and the future. In classification in BioApps (pp. 323–350). Springer, Cham, https://doi.org/10.1007/978-3-319-65981-7_12.

  29. Ronneberger O., Fischer P., Brox T. (2015) U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture notes in computer science, vol 9351. Springer, Cham https://doi.org/10.1007/978-3-319-24574-4_28

  30. Roy, A. G., Navab, N., &Wachinger, C. (2018). Concurrent spatial and channel' squeeze & excitation in fully convolutional networks. In international conference on medical image computing and computer-assisted intervention (pp. 421-429). Springer, Cham, arXiv:1803.02579.

  31. Rundo L, Han C, Nagano Y, Zhang J, Hataya R, Militello C, Cazzaniga P (2019) USE-net: incorporating squeeze-and-excitation blocks into U-net for prostate zonal segmentation of multi-institutional MRI datasets. Neurocomputing 365:31–43

    Article  Google Scholar 

  32. Schlemper J, Oktay O, Schaap M, Heinrich M, Kainz B, Glocker B, Rueckert D (2019) Attention gated networks: learning to leverage salient regions in medical images. Med Image Anal 53:197–207

    Article  Google Scholar 

  33. Shorten C, Khoshgoftaar TM (2019) A survey on image data augmentation for deep learning. J Big Data 6(1):1–48

    Article  Google Scholar 

  34. Siddique, N., Sidike, P., Elkin, C., &Devabhaktuni, V. (2020). U-net and its variants for medical image segmentation: theory and applications. arXiv preprint arXiv:2011.01118, https://doi.org/10.1109/ACCESS.2021.3086020

  35. Singh, N., Bansal, D., & Nagpal, D. (2020). Deep learning-based retinal vessel segmentation: a review. Adv. Math., Sci. J, 9(6), 3827-3837, DOI: https://doi.org/10.37418/amsj.9.6.62

  36. Soomro TA, Afifi AJ, Zheng L, Soomro S, Gao J, Hellwich O, Paul M (2019) Deep learning models for retinal blood vessels segmentation: a review. IEEE Access 7:71696–71717

    Article  Google Scholar 

  37. Tan Z, Simkin S, Lai C, Dai S (2019) A deep learning algorithm for automated diagnosis of retinopathy of prematurity plus disease. Trans Vision SciTechnol 8(6):23–23

    Article  Google Scholar 

  38. Taylor S, Brown JM, Gupta K, Campbell JP, Ostmo S, Chan RP, Kalpathy-Cramer J (2019) Monitoring disease progression with a quantitative severity scale for retinopathy of prematurity using deep learning. JAMA Ophthalmol 137(9):1022–1028

    Article  Google Scholar 

  39. Tiwari, S. S., Dholaria, A., Pandey, R., Nigam, G., Agrawal, R., Walambe, R., & Kotecha, K. (2020). Deep learning-based framework for retinal vasculature segmentation. In congress on intelligent systems (pp. 275-290). Springer, Singapore.

  40. Tong Y, Lu W, Deng QQ, Chen C, Shen Y (2020) Automated identification of retinopathy of prematurity by image-based deep learning. Eye Vision 7(1):1–12

    Article  Google Scholar 

  41. Uysal E, Güraksin GE (2021) Computer-aided retinal vessel segmentation in retinal images: convolutional neural networks. Multimed Tools Appl 80(3):3505–3528

    Article  Google Scholar 

  42. Vijayalakshmi C, Sakthivel P, Vinekar A (2020) Automated detection and classification of telemedical retinopathy of prematurity images. Telemed e-Health 26(3):354–358. https://doi.org/10.1089/tmj.2019.0004

    Article  Google Scholar 

  43. Wang, J., Lv, P., Wang, H., & Shi, C. (2021). SAR-U-net: squeeze-and-excitation block and atrous spatial pyramid pooling based residual U-net for automatic liver CT segmentation. arXiv preprint arXiv:2103.06419.

  44. Yu Y, Zhu H (2021) Retinal vessel segmentation with constrained-based nonnegative matrix factorization and 3D modified attention U-net. EURASIP J Image Video Proces 2021(1):1–21

    Article  Google Scholar 

Download references

Acknowledgments

We thank Dr. Nilesh Giri, Dr. Pravin Hankare, and Dr. Anita Gaikwad from H. V. Desai Eye Hospital, Pune, for providing annotated images for research. We acknowledge H. V. Desai Eye Hospital staff’s help in providing daily fundus images to extend our current work. We also thank Mr. Anup Agrawal for helping in ground truth preparation using Adobe Photoshop.

Availability of data and material

PBMA’s H. V. Desai Eye Hospital, Pune.

Code availability

Available with authors—custom code.

Author information

Authors and Affiliations

  1. Symbiosis Institute of Technology (SIT), Symbiosis International (Deemed) University (SIU), Pune, India

    Ranjana Agrawal, Rahee Walambe & Ketan Kotecha

  2. Dr. Vishwanath Karad MIT World Peace University, Pune, India

    Ranjana Agrawal

  3. PBMA’s H. V. Desai Eye Hospital, Pune, India

    Sucheta Kulkarni & Madan Deshpande

  4. Symbiosis Centre for Applied Artificial Intelligence (SCAAI), Symbiosis International (Deemed) University, Pune, India

    Rahee Walambe & Ketan Kotecha

Authors
  1. Ranjana Agrawal
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sucheta Kulkarni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Rahee Walambe
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Madan Deshpande
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Ketan Kotecha
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Ranjana Agrawal: Main author, creativity, programming, and system design, main conceptual work. Dr. Sucheta Kulkarni: Domain expert, guidance about the disease-specific concepts, dataset collection, and validation of results. Rahee Walambe: System design, conceptualization and ideation, and paper writing and review. Col. Madan Deshpande: Critical review of the manuscript. Ketan Kotecha: Conceptualization and preliminary assessment of work, paper review, senior author.

Corresponding author

Correspondence to Ketan Kotecha.

Ethics declarations

There is nothing to declare by the authors.

Research involving human participants or animals

This study does not contain any studies with human participants or animals performed by any author.

Informed consent

Images obtained from preterm babies enrolled in the hospital’s screening program were used anonymously (without disclosing identity). As a protocol, written informed consent regarding the use of data for quality assurance and research purposes is obtained from preterm babies’ parents before screening for ROP.

Conflicts of interest/competing interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Agrawal, R., Kulkarni, S., Walambe, R. et al. Deep dive in retinal fundus image segmentation using deep learning for retinopathy of prematurity. Multimed Tools Appl 81, 11441–11460 (2022). https://doi.org/10.1007/s11042-022-12396-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12396-z

Keywords

Search

Navigation