Skip to main content

Advertisement

SpringerLink
Log in

Segmentation of retinal vessels in fundus images based on U-Net with self-calibrated convolutions and spatial attention modules

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Automated and accurate segmentation of retinal vessels in fundus images is an important step for screening and diagnosing various ophthalmologic diseases. However, many factors, including the variations of vessels in color, shape and size, make this task become an intricate challenge. One kind of the most popular methods for vessel segmentation is U-Net based methods. However, in the U-Net based methods, the size of the convolution kernels is generally fixed. As a result, the receptive field for an individual convolution operation is single, which is not conducive to the segmentation of retinal vessels with various thicknesses. To overcome this problem, in this paper, we employed self-calibrated convolutions to replace the traditional convolutions for the U-Net, which can make the U-Net learn discriminative representations from different receptive fields. Besides, we proposed an improved spatial attention module, instead of using traditional convolutions, to connect the encoding part and decoding part of the U-Net, which can improve the ability of the U-Net to detect thin vessels. The proposed method has been tested on Digital Retinal Images for Vessel Extraction (DRIVE) database and Child Heart and Health Study in England Database (CHASE DB1). The metrics used to evaluate the performance of the proposed method are accuracy (ACC), sensitivity (SE), specificity (SP), F1-score (F1) and the area under the receiver operating characteristic curve (AUC). The ACC, SE, SP, F1 and AUC obtained by the proposed method are 0.9680, 0.8036, 0.9840, 0.8138 and 0.9840 respectively on DRIVE database, and 0.9756, 0.8118, 0.9867, 0.8068 and 0.9888 respectively on CHASE DB1, which are better than those obtained by the traditional U-Net (the ACC, SE, SP, F1 and AUC obtained by U-Net are 0.9646, 0.7895, 0.9814, 0.7963 and 0.9791 respectively on DRIVE database, and 0.9733, 0.7817, 0.9862, 0.7870 and 0.9810 respectively on CHASE DB1). The experimental results indicate that the proposed modifications in the U-Net are effective for vessel segmentation.

The structure of the proposed network

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Alom MZ, Yakopcic C, Hasan M, Taha TM, Asari VK (2019) Recurrent residual U-Net for medical image segmentation. J Med Imaging 6(1):014,006

    Article  Google Scholar 

  2. Azzopardi G, Strisciuglio N, Vento M, Petkov N (2015) Trainable cosfire filters for vessel delineation with application to retinal images. Med Image Anal 19(1):46–57

    Article  PubMed  Google Scholar 

  3. Fan Z, Lu J, Wei C, Huang H, Cai X, Chen X (2018) A hierarchical image matting model for blood vessel segmentation in fundus images. IEEE Trans Image Process 28(5):2367–2377

    Article  Google Scholar 

  4. Fazlali HR, Karimi N, Soroushmehr SR, Shirani S, Nallamothu BK, Ward KR, Samavi S, Najarian K (2018) Vessel segmentation and catheter detection in x-ray angiograms using superpixels. Med Biol Eng Comput 56(9):1515–1530

    Article  PubMed  Google Scholar 

  5. Fraz MM, Remagnino P, Hoppe A, Uyyanonvara B, Rudnicka AR, Owen CG, Barman SA (2012) Blood vessel segmentation methodologies in retinal images–a survey. Comput Methods Programs Biomed 108(1):407–433

    Article  CAS  PubMed  Google Scholar 

  6. Fraz MM, Remagnino P, Hoppe A, Uyyanonvara B, Rudnicka AR, Owen CG, Barman SA (2012) An ensemble classification-based approach applied to retinal blood vessel segmentation. IEEE Trans Biomed Eng 59(9):2538–2548

    Article  PubMed  Google Scholar 

  7. Guo C, Szemenyei M, Yi Y, Wang W, Chen B, Fan C (2020) SA-UNet: Spatial attention U-Net for retinal vessel segmentation. arXiv:2004.03696

  8. Guo X, Xiao R, Zhang T, Chen C, Wang J, Wang Z (2020) A novel method to model hepatic vascular network using vessel segmentation, thinning, and completion. Med Biol Eng Comput 1–16

  9. Hamamoto Y, Uchimura S, Watanabe M, Yasuda T, Mitani Y, Tomita S (1998) A Gabor filter-based method for recognizing handwritten numerals. Pattern Recogn 31(4):395–400

    Article  Google Scholar 

  10. Lam BS, Gao Y, Liew AWC (2010) General retinal vessel segmentation using regularization-based multiconcavity modeling. IEEE Trans Med Imaging 29(7):1369–1381

    Article  PubMed  Google Scholar 

  11. Lecun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521(7553):436–444

    Article  CAS  PubMed  Google Scholar 

  12. Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Article  Google Scholar 

  13. Li Q, Feng B, Xie L, Liang P, Zhang H, Wang T (2015) A cross-modality learning approach for vessel segmentation in retinal images. IEEE Trans Med Imaging 35(1):109–118

    Article  PubMed  Google Scholar 

  14. Li X, Wang L, Sung E (2008) AdaBoost with SVM-based component classifiers. Eng Appl Artif Intel 21(5):785–795

    Article  Google Scholar 

  15. Liskowski P, Krawiec K (2016) Segmenting retinal blood vessels with deep neural networks. IEEE Trans Med Imaging 35(11):2369–2380

    Article  PubMed  Google Scholar 

  16. Liu I, Sun Y (1993) Recursive tracking of vascular networks in angiograms based on the detection-deletion scheme. IEEE Trans Med Imaging 12(2):334–341

    Article  CAS  PubMed  Google Scholar 

  17. Liu JJ, Hou Q, Cheng MM, Wang C, Feng J (2020) Improving convolutional networks with self-calibrated convolutions. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 10,096–10,105

  18. Mendonca AM, Campilho A (2006) Segmentation of retinal blood vessels by combining the detection of centerlines and morphological reconstruction. IEEE Trans Med Imaging 25(9):1200–1213

    Article  PubMed  Google Scholar 

  19. Mou L, Zhao Y, Fu H, Liu Y, Cheng J, Zheng Y, Su P, Yang J, Chen L, Frangi AF et al (2021) CS2-Net: Deep learning segmentation of curvilinear structures in medical imaging. Med Image Anal 67:101,874

    Article  Google Scholar 

  20. Nguyen V, Blumenstein M (2011) An application of the 2D gaussian filter for enhancing feature extraction in off-line signature verification. In: 2011 international conference on document analysis and recognition. IEEE, pp 339–343

  21. Ricci E, Perfetti R (2007) Retinal blood vessel segmentation using line operators and support vector classification. IEEE Trans Med Imaging 26(10):1357–1365

    Article  PubMed  Google Scholar 

  22. Ronneberger O, Fischer P, Brox T (2015) U-Net: Convolutional networks for biomedical image segmentation. In: International Conference on Medical image computing and computer-assisted intervention. Springer, pp 234–241

  23. Roychowdhury S, Koozekanani DD, Parhi KK (2014) Blood vessel segmentation of fundus images by major vessel extraction and subimage classification. IEEE J Biomed Health Inf 19(3):1118–1128

    Google Scholar 

  24. Salem SA, Salem NM, Nandi AK (2007) Segmentation of retinal blood vessels using a novel clustering algorithm (RACAL) with a partial supervision strategy. Med Biol Eng Comput 45(3):261–273

    Article  PubMed  Google Scholar 

  25. Staal J, Abràmoff MD, Niemeijer M, Viergever MA, Van Ginneken B (2004) Ridge-based vessel segmentation in color images of the retina. IEEE Trans Med Imaging 23(4):501–509

    Article  PubMed  Google Scholar 

  26. Wang B, Qiu S, He H (2019) Dual encoding U-Net for retinal vessel segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 84–92

  27. Wang B, Wang S, Qiu S, Wei W, Wang H, He H (2020) CSU-Net: a context spatial u-net for accurate blood vessel segmentation in fundus images. IEEE J Biomed Health Inform 25(4):1128–1138

    Article  Google Scholar 

  28. Wang W, Wang W, Hu Z (2019) Segmenting retinal vessels with revised top-bottom-hat transformation and flattening of minimum circumscribed ellipse. Med Biol Eng Comput 57(7):1481–1496

    Article  PubMed  Google Scholar 

  29. Wei J, Zhu G, Fan Z, Liu J, Rong Y, Mo J, Li W, Chen X (2022) Genetic U-Net: automatically designed deep networks for retinal vessel segmentation using a genetic algorithm. IEEE Trans Med Imaging 41(2):292–307

    Article  PubMed  Google Scholar 

  30. Woo S, Park J, Lee JY, Kweon IS (2018) CBAM: Convolutional block attention module. In: Proceedings of the European conference on computer vision (ECCV), pp 3–19

  31. Wu Y, Xia Y, Song Y, Zhang Y, Cai W (2018) Multiscale network followed network model for retinal vessel segmentation. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 119–126

  32. Zhao Y, Rada L, Chen K, Harding SP, Zheng Y (2015) Automated vessel segmentation using infinite perimeter active contour model with hybrid region information with application to retinal images. IEEE Trans Med Imaging 34(9):1797–1807

    Article  PubMed  Google Scholar 

Download references

Funding

This work was supported in part by the National Key R&D Program of China under grant 2021ZD0111502, in part by the Science and Technology Planning Project of Guangdong Province of China under grant 180917144960530, in part by the Project of Educational Commission of Guangdong Province of China under grant 2017KZDXM032, in part by Science Research Startup Foundation of Shantou University NTF20021, in part by the State Key Lab of Digital Manufacturing Equipment and Technology under grant DMETKF2019020, in part by the Project of Robot Automatic Design Platform combining Multi-Objective Evolutionary Computation and Deep Neural Network under grant 2019A050519008, in part by the Guangdong Natural Science Foundation under grant 2022A1515011396.

Author information

Authors and Affiliations

  1. Department of Electronic and Information Engineering, Shantou University, 515063, Guangdong, China

    YiBiao Rong, Yu Xiong, Chong Li, Ying Chen, Peiwei Wei, Chuliang Wei & Zhun Fan

  2. Key Lab of Digital Signal and Image Processing of Guangdong Province, Shantou University, 515063, Guangdong, China

    YiBiao Rong, Yu Xiong, Chong Li, Ying Chen, Peiwei Wei, Chuliang Wei & Zhun Fan

  3. Department of Microbiology and Immunology, Shantou University Medical College, Guangdong, 515041, China

    Peiwei Wei

Authors
  1. YiBiao Rong
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yu Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ying Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Peiwei Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Chuliang Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Zhun Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhun Fan.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Rong, Y., Xiong, Y., Li, C. et al. Segmentation of retinal vessels in fundus images based on U-Net with self-calibrated convolutions and spatial attention modules. Med Biol Eng Comput 61, 1745–1755 (2023). https://doi.org/10.1007/s11517-023-02806-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02806-1

Keywords

Search

Navigation