Skip to main content
SpringerLink
Log in

Multiple lesion segmentation in diabetic retinopathy with dual-input attentive RefineNet

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

To address the issue of complex structure, various sizes and the interclass similarity of different lesions, this paper proposes a dual-input attentive RefineNet (DARNet) for automatic multiple lesion segmentation of diabetic retinopathy. DARNet includes a global image encoder, local image encoder and attention refinement decoder. The whole image and the patch image are used as the dual input and fed into ResNet50 and ResNet101 for down-sampling, respectively. The high-level attention refinement decoder adopts a dual attention mechanism to integrate the same-level features in the two encoders with the output of the low-level attention refinement module for multiscale feature fusion, which focuses the model on the lesion area to generate accurate predictions. We evaluated the segmentation performance of four lesions on three datasets, and the proposed method reached an average accuracy of 0.9582/0.9617/0.9578 and a dice score of 0.9521/0.9637/0.9508 on IDRiD, E-ophtha and DDR. Extensive experimental results demonstrate the proposed DARNet outperforms the state- of-the-art models and has better robustness and accuracy. It not only preserves the contour details and shape features of multiscale lesions, but also overcomes the interference of similar tissues and noises to realize accurate multi-lesion segmentation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Availability of Data and Materials

Data related to the current study are available from the corresponding author on reasonable request.

Code Availability

The codes used during the study are available from the corresponding author by request.

References

  1. Ting DSW, Cheung GCM, Wong TY (2016) Diabetic retinopathy: global prevalence, major risk factors, screening practices and public health challenges: a review. Clin Exper Ophthalmol 44(4):260–277

    Article  Google Scholar 

  2. Yau JWY, Rogers SL, Kawasaki R, Lamoureux EL, Kowalski JW, Bek T, Chen S-J, Dekker JM, Fletcher A, Grauslund J et al (2012) Global prevalence and major risk factors of diabetic retinopathy. Diabetes Care 35(3):556–564

    Article  Google Scholar 

  3. Salamat N, Missen MMS, Rashid A (2019) Diabetic retinopathy techniques in retinal images: A review. Artif Intell Med 97:168–188

    Article  Google Scholar 

  4. Stolte S, Fang R (2020) A survey on medical image analysis in diabetic retinopathy. Med Image Anal 64:101742

    Article  Google Scholar 

  5. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22):2402–2410

    Article  Google Scholar 

  6. Asiri N, Hussain M, Al Adel F, Alzaidi N (2019) Deep learning based computer-aided diagnosis systems for diabetic retinopathy: A survey. Artif Intell Med 99:101701

    Article  Google Scholar 

  7. Zhang W, Dai Y, Liu M, Chen Y, Zhong J, Yi Z (2021) Deepuwf-plus: automatic fundus identification and diagnosis system based on ultrawide-field fundus imaging. Appl Intell 51:7533–7551

    Article  Google Scholar 

  8. Koh JEW, Ng EYK, Bhandary SV, Laude A, Acharya UR (2018) Automated detection of retinal health using phog and surf features extracted from fundus images. Appl Intell 48:1379–1393

    Google Scholar 

  9. Acharya UR, Mookiah MRK, Koh Joel EW, Tan JH, Bhandary SV, Rao AK, Fujita H, Hagiwara Y, Chua CK, Laude A (2016) Automated screening system for retinal health using bi-dimensional empirical mode decomposition and integrated index. Comput Biol Med 75:54–62

    Article  Google Scholar 

  10. Xu Y, Zhou Z, Li X, Zhang N, Zhang M, Wei P (2021) Ffu-net: Feature fusion u-net for lesion segmentation of diabetic retinopathy. BioMed Res Int 2021

  11. Foo A, Hsu W, Lee ML, Lim G, Wong TY (2020) Multi-task learning for diabetic retinopathy grading and lesion segmentation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 13267–13272

  12. Yang Y, Li T, Li W, Wu H, Fan W, Zhang W (2017) Lesion detection and grading of diabetic retinopathy via two-stages deep convolutional neural networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 533–540

  13. Sambyal N, Saini P, Syal R, Gupta V (2020) Modified u-net architecture for semantic segmentation of diabetic retinopathy images. Biocybern Biomed Eng 40(3):1094–1109

    Article  Google Scholar 

  14. Guo S, Li T, Kang H, Li N, Zhang Y, Wang K (2019) L-seg: An end-to-end unified framework for multi-lesion segmentation of fundus images. Neurocomputing 349:52–63

    Article  Google Scholar 

  15. Mo J, Zhang L, Feng Y (2018) Exudate-based diabetic macular edema recognition in retinal images using cascaded deep residual networks. Neurocomputing 290:161–171

    Article  Google Scholar 

  16. Porwal P, Pachade S, Kamble R, Kokare M, Deshmukh G, Sahasrabuddhe V, Meriaudeau F (2018) Indian diabetic retinopathy image dataset (idrid): a database for diabetic retinopathy screening research. Data 3(3):25

    Article  Google Scholar 

  17. Decenciere E, Cazuguel G, Zhang X, Thibault G, Klein J-C, Meyer F, Marcotegui B, Quellec G, Lamard M, Danno R et al (2013) Teleophta: Machine learning and image processing methods for teleophthalmology. Irbm 34(2):196–203

    Article  Google Scholar 

  18. Li T, Gao Y, Wang K, Guo S, Liu H, Kang H (2019) Diagnostic assessment of deep learning algorithms for diabetic retinopathy screening. Inf Sci 501:511–522

    Article  Google Scholar 

  19. Wu B, Zhu W, Shi F, Zhu S, Chen X (2017) Automatic detection of microaneurysms in retinal fundus images. Comput Med Imaging Graph 55:106–112

    Article  Google Scholar 

  20. Long S, Huang X, Chen Z, Pardhan S, Zheng D (2019) Automatic detection of hard exudates in color retinal images using dynamic threshold and svm classification: algorithm development and evaluation. BioMed Res Int 2019

  21. Colomer A, Igual J, Naranjo V (2020) Detection of early signs of diabetic retinopathy based on textural and morphological information in fundus images. Sensors 20(4):1005

    Article  Google Scholar 

  22. Kälviäinen RVJPH, Uusitalo H (2007) Diaretdb1 diabetic retinopathy database and evaluation protocol. In: Medical image understanding and analysis, vol 2007. Citeseer, p 61

  23. Amin J, Sharif M, Yasmin M, Ali H, Fernandes S L (2017) A method for the detection and classification of diabetic retinopathy using structural predictors of bright lesions. J Comput Sci 19:153–164

    Article  Google Scholar 

  24. Zhou Y, He X, Huang L, Liu L, Zhu F, Cui S, Shao L (2019) Collaborative learning of semi-supervised segmentation and classification for medical images. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp 2079–2088

  25. Xue J, Yan S, Qu J, Qi F, Qiu C, Zhang H, Chen M, Liu T, Li D, Liu X (2019) Deep membrane systems for multitask segmentation in diabetic retinopathy. Knowl-Based Syst 183:104887

    Article  Google Scholar 

  26. Tan JH, Fujita H, Sivaprasad S, Bhandary SV, Rao AK, Chua KC, Acharya UR (2017) Automated segmentation of exudates, haemorrhages, microaneurysms using single convolutional neural network. Inf Sci 420:66–76

    Article  Google Scholar 

  27. Sun J, Peng Y, Guo Y, Li D (2021) Segmentation of the multimodal brain tumor image used the multi-pathway architecture method based on 3d fcn. Neurocomputing 423:34– 45

    Article  Google Scholar 

  28. Ting DSW, Cheung CY-L, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H, Garcia-Franco R, San Yeo IY, Lee S Y et al (2017) Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. Jama 318(22):2211–2223

    Article  Google Scholar 

  29. Decencière E, Zhang X, Cazuguel G, Lay B, Cochener B, Trone C, Gain P, Ordonez R, Massin P, Erginay A et al (2014) Feedback on a publicly distributed image database: the messidor database. Image Anal Stereol 33(3):231–234

    Article  Google Scholar 

  30. Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403

  31. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H (2018) Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European conference on computer vision (ECCV), pp 801–818

  32. Zhou Y, Wang B, Huang L, Cui S, Shao L (2021) A benchmark for studying diabetic retinopathy: Segmentation, grading, and transferability. IEEE Trans Med Imaging 40(3):818–828

    Article  Google Scholar 

  33. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  34. Lin G, Liu F, Milan A, Shen C, Reid I (2019) Refinenet: Multi-path refinement networks for dense prediction. IEEE Trans Pattern Anal Mach Intell 42(5):1228–1242

    Google Scholar 

  35. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: A large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition. IEEE, pp 248–255

  36. Ronneberger O, Fischer P, Brox T (2015) U-net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  37. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Anal Mach Intell 39(12):2481–2495

    Article  Google Scholar 

  38. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations

Download references

Acknowledgements

The authors gratefully acknowledge the financial supports by the National Science Foundation of China under Grant number 61976126, as well as the Shandong Nature Science Foundation of China under Grant numbers ZR2019MF003, ZR2020MF132, ZR2020MH291.

Funding

This work was supported in part by the National Natural Science Foundation of China (Grant No. 61976126), Shandong Nature Science Foundation of China (No. ZR2019MF003, ZR2020MF132, ZR2020MH291).

Author information

Authors and Affiliations

  1. College of Computer Science and Engineering, Shandong University of Science and Technology, Qingdao, 266590, Shandong, China

    Yanfei Guo & Yanjun Peng

  2. Shandong Province Key Laboratory of Wisdom Mining Information Technology, Shandong University of Science and Technology, Qingdao, 266590, Shandong, China

    Yanjun Peng

Authors
  1. Yanfei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yanjun Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yanfei Guo: Conceptualization, Methodology, Software, Writing the original draft. Yanjun Peng: Data curation, Supervision, Funding acquisition, Project administration.

Corresponding author

Correspondence to Yanjun Peng.

Ethics declarations

Conflicts of Interest

The authors declare there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, Y., Peng, Y. Multiple lesion segmentation in diabetic retinopathy with dual-input attentive RefineNet. Appl Intell 52, 14440–14464 (2022). https://doi.org/10.1007/s10489-022-03204-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03204-0

Keywords

Search

Navigation