Skip to main content
SpringerLink
Log in

Encoding-decoding Network With Pyramid Self-attention Module For Retinal Vessel Segmentation

  • Research Article
  • Pattern Recognition
  • Published:
International Journal of Automation and Computing Aims and scope Submit manuscript

Abstract

Retina vessel segmentation is a vital step in diagnosing ophthalmologic diseases. Traditionally, ophthalmologists segment retina vessels by hand, which is time-consuming and error-prone. Thus, more and more researchers are committed to the research of automatic segmentation algorithms. With the development of convolution neural networks (CNNs), many tasks can be solved by CNNs. In this paper, we propose an encoding-decoding network with a pyramid self-attention module (PSAM) to segment retinal vessels. The network follows a U shape structure, and it comprises stacked feature selection blocks (FSB) and a PSAM. The proposed FSB consists of two convolution blocks with the same weight and a channel-wise attention block. At the head of the network, we apply a PSAM consisting of three parallel self-attention modules to capture long-range dependence of different scales. Due to the power of PSAM and FSB, the performance of the network improves. We have evaluated our model on two public datasets: DRIVE and CHASE_DB1. The results show the performance of our model is better than other methods. The F1, Accuracy, and area under curve (AUC) are 82.21%/80.57%, 95.65%/97.02%, and 98.16%/98.46% on DRIVE and CHASE_DB1, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. C. Kirbas, F. Quek. A review of vessel extraction techniques and algorithms. ACM Computing Surveys, vol. 36, no. 2, pp. 81–121, 2004. DOI: https://doi.org/10.1145/1031120.1031121.

    Article  Google Scholar 

  2. M. M. Fraz, P. Remagnino, A. Hoppe, B. Uyyanonvara, A. R. Rudnicka, C. G. Owen, S. A. Barman. Blood vessel segmentation methodologies in retinal images-a survey. Computer Methods and Programs in Biomedicine, vol. 108, no. 1, pp. 407–433, 2012. DOI: https://doi.org/10.1016/j.cmpb.2012.03.009.

    Article  Google Scholar 

  3. B. Al-Diri, A. Hunter, D. Steel. An active contour model for segmenting and measuring retinal vessels. IEEE Transactions on Medical Imaging, vol. 28, no. 9, pp. 1488–1497, 2009. DOI: https://doi.org/10.1109/TMI.2009.2017941.

    Article  Google Scholar 

  4. G. Azzopardi, N. Petkov. Automatic detection of vascular bifurcations in segmented retinal images using trainable COSFIRE filters. Pattern Recognition Letters, vol. 34, no. 8, pp. 922–933, 2013. DOI: https://doi.org/10.1016/j.patrec.2012.11.002.

    Article  Google Scholar 

  5. S. Aslani, H. Sarnel. A new supervised retinal vessel segmentation method based on robust hybrid features. Biomedical Signal Processing and Control, vol. 30, pp. 1–12, 2016. DOI: https://doi.org/10.1016/j.bspc.2016.05.006.

    Article  Google Scholar 

  6. L. Wang, A. Bhalerao, R. Wilson. Analysis of retinal vasculature using a multiresolution hermite model. IEEE Transactions on Medical Imaging, vol. 26, no. 2, pp. 137–152, 2007. DOI: https://doi.org/10.1109/TMI.2006.889732.

    Article  Google Scholar 

  7. Y. Yin, M. Adel, S. Bourennane. Retinal vessel segmentation using a probabilistic tracking method. Pattern Recognition, vol. 45, no. 4, pp. 1235–1244, 2012. DOI: https://doi.org/10.1016/j.patcog.2011.09.019.

    Article  Google Scholar 

  8. Q. Qi, Q. D. Li, Y. Q. Cheng, Q. Q. Hong. Skeleton marching-based parallel vascular geometry reconstruction using implicit functions. International Journal of Automation and Computing, vol. 17, no. 1, pp. 30–43, 2020. DOI: https://doi.org/10.1007/s11633-019-1189-4.

    Article  Google Scholar 

  9. W. J. Bai, M. Sinclair, G. Tarroni, O. Oktay, M. Rajchl, G. Vaillant, A. M. Lee, N. Aung, E. Lukaschuk, M. M. Sanghvi, F. Zemrak, K. Fung, J. M. Paiva, V. Carapella, Y. J. Kim, H. Suzuki, B. Kainz, P. M. Matthews, S. E. Petersen, S. K. Piechnik, S. Neubauer, B. Glocker, D. Rueckert. Human-level CMR image analysis with deep fully convolutional networks. [Online], Available: https://arxiv.org/abs/1710.09289v2, 2017.

  10. Z. W. He, L. Zhang, F. Y. Liu. DiscoStyle: Multi-level logistic ranking for personalized image style preference inference. International Journal of Automation and Computing, vol. 17, no. 5, pp. 637–651, 2020. DOI: https://doi.org/10.1007/s11633-020-1244-1.

    Article  Google Scholar 

  11. Y. R. Liang, Z. Y. Xiao. Image encryption algorithm based on compressive sensing and fractional DCT via polynomial interpolation. International Journal of Automation and Computing, vol. 17, no. 2, pp. 292–304, 2020. DOI: https://doi.org/10.1007/s11633-018-1159-2.

    Article  Google Scholar 

  12. F. Z. Liao, M. Liang, Z. Li, X. L. Hu, S. Song. Evaluate the malignancy of pulmonary nodules using the 3-D deep leaky noisy-OR network. IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 11, pp. 3484–3495, 2019. DOI: https://doi.org/10.1109/TNNLS.2019.2892409.

    Article  Google Scholar 

  13. P. Moeskops, M. Veta, M. W. Lafarge, K. A. J. Eppenhof, J. P. W. Pluim. Adversarial training and dilated convolutions for brain MRI segmentation. In Proceedings of the 3rd International Workshop on Deep Learning in Medical Image Analysis and the 7th International Workshop on Multimodal Learning for Clinical Decision Support, Springer, Quebec City, Canada, pp. 56–64, 2017. DOI: https://doi.org/10.1007/978-3-319-67558-9_7.

    Google Scholar 

  14. S. Wang, M. Zhou, Z. Y. Liu, Z. Y. Liu, D. S. Gu, Y. L. Zang, D. Dong, O. Gevaert, J. Tian. Central focused convolutional neural networks: Developing a data-driven model for lung nodule segmentation. Medical Image Analysis, vol. 40, pp. 172–183, 2017. DOI: https://doi.org/10.1016/j.media.2017.06.014.

    Article  Google Scholar 

  15. Z. Y. Liu. Retinal vessel segmentation based on fully convolutional networks. [Online], Available: https://arxiv.org/abs/1911.09915, 2019.

  16. Z. W. Gu, J. Cheng, H. Z. Fu, K. Zhou, H. Y. Hao, Y. T. Zhao, T. Y. Zhang, S. H. Gao, J. Liu. CE-Net: Context encoder network for 2D medical image segmentation. IEEE Transactions on Medical Imaging, vol. 38, no. 10, pp. 2281–2292, 2019. DOI: https://doi.org/10.1109/TMI.2019.2903562.

    Article  Google Scholar 

  17. L. Luo, D. Y. Xue, X. L. Feng. HybridNetSeg: A compact hybrid network for retinal vessel segmentation. [Online], Available: https://arxiv.org/abs/1911.09982, 2019.

  18. J. Son, S. J. Park, K. H. Jung. Retinal vessel segmentation in fundoscopic images with generative adversarial networks. [Online], Available: https://arxiv.org/abs/1706.09318, 2017.

  19. J. Hu, L. Shen, S. Albanie, G. Sun, E. H. Wu. Squeeze-and-excitation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 42, no. 8, pp. 2011–2023, 2020. DOI: https://doi.org/10.1109/TPAMI.2019.2913372.

    Article  Google Scholar 

  20. J. Fu, J. Liu, H. J. Tian, Y. Li, Y. J. Bao, Z. W. Fang, H. Q. Lu. Dual attention network for scene segmentation. In Proceedings of IEEE/CVF Conference on Computer Vison and Pattern Recognition, IEEE, Long Beach, USA, pp. 3141–3149, 2019. DOI: https://doi.org/10.1109/CVPR.2019.00326.

    Google Scholar 

  21. E. Shelhamer, J. Long, T. Darrell. Fully convolutional networks for semantic segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 39, no. 4, pp. 640–651, 2017. DOI: https://doi.org/10.1109/TPAMI.2016.2572683.

    Article  Google Scholar 

  22. O. Ronneberger, P. Fischer, T. Brox. U-Net: Convolutional networks for biomedical image segmentation. In Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention, Springer, Munich, Germany, pp. 234–241, 2015. DOI: https://doi.org/10.1007/978-3-319-24574-4_28.

    Google Scholar 

  23. N. Abraham, N. M. Khan. A novel focal Tversky loss function with improved attention U-Net for lesion segmentation. [Online], Available: https://arxiv.org/abs/1810.07842, 2018.

  24. M. Z. Alom, M. Hasan, C. Yakopcic, T. M. Taha, V. K. Asari. Recurrent residual convolutional neural network based on U-Net (R2U-Net) for medical image segmentation. [Online], Available: https://arxiv.org/abs/1802.06955, 2018.

  25. H. Z. Fu, J. Cheng, Y. W. Xu, D. W. K. Wong, J. Liu, X. C. Cao. Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. IEEE Transactions on Medical Imaging, vol. 37, no. 7, pp. 1597–1605, 2018. DOI: https://doi.org/10.1109/TMI.2018.2791488.

    Article  Google Scholar 

  26. L. C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, A. L. Yuille. DeepLab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 40, no. 4, pp. 834–848, 2018. DOI: https://doi.org/10.1109/TPAMI.2017.2699184.

    Article  Google Scholar 

  27. Q. Zhang, Z. P. Cui, X. G. Niu, S. J. Geng, Y. Qiao. Image segmentation with pyramid dilated convolution based on ResNet and U-Net. In Proceedings of the 24th International Conference International Conference on Neural Information Processing, Springer, Guangzhou, China, pp. 364–372, 2017. DOI: https://doi.org/10.1007/978-3-319-70096-0_38.

    Google Scholar 

  28. H. Y. Xia, W. F. Sun, S. X. Song, X. W. Mou. Md-Net: Multi-scale dilated convolution network for CT images segmentation. Neural Processing Letters, vol. 51, no. 3, pp. 2915–2927, 2020. DOI: https://doi.org/10.1007/s11063-020-10230-x.

    Article  Google Scholar 

  29. D. G. Xiao, P. Zhong. Image semantic segmentation using deep convolutional nets, fully connected conditional random fields, and dilated convolution. In Proceedings of the 21st IEEE International Conference on High Performance Computing and Communications; the 17th IEEE International Conference on Smart City; the 5th IEEE International Conference on Data Science and Systems, IEEE, Zhangjiajie, China, pp. 1872–1877, 2019. DOI: https://doi.org/10.1109/HPCC/SmartCity/DSS.2019.00258.

    Google Scholar 

  30. X. L. Wang, R. Girshick, A. Gupta, K. M. He. Non-local neural networks. [Online], Available: https://arxiv.org/abs/1711.07971, 2017.

  31. H. Tang, H. Liu, D. Xu, P. H. S. Torr, N. Sebe. AttentionGAN: Unpaired image-to-image translation using attention-guided generative adversarial networks. [Online], Available: https://arxiv.org/abs/1911.11897, 2019.

  32. B. W. Li, X. J. Qi, T. Lukasiewicz, P. H. S. Torr. Controllable text-to-image generation. [Online], Available: https://arxiv.org/abs/1909.07083, 2019.

  33. O. Oktay, J. Schlemper, L. L. Folgoc, M. Lee, M. Heinrich, K. Misawa, K. Mori, S. McDonagh, N. Y. Hammerla, B. Kainz, B. Glocker, D. Rueckert. Attention U-Net: Learning where to look for the pancreas. [Online], Available: https://arxiv.org/abs/1804.03999, 2018.

  34. A. D. Hoover, V. Kouznetsova, M. Goldbaum. Locating blood vessels in retinal images by piecewise threshold probing of a matched filter response. IEEE Transactions on Medical Imaging, vol. 19, no. 3, pp. 203–210, 2000. DOI: https://doi.org/10.1109/42.845178.

    Article  Google Scholar 

  35. M. Niemeijer, J. Staal, B. Van Ginneken, M. Loog, M. D. Abràmoff. Comparative study of retinal vessel segmentation methods on a new publicly available database. In Proceedings of SPIE 5370, Medical Imaging 2004: Image Processing, San Diego, USA, pp. 648–656, 2004. DOI: https://doi.org/10.1117/12.535349.

  36. S. Roychowdhury, D. D. Koozekanani, K. K. Parhi. Blood vessel segmentation of fundus images by major vessel extraction and subimage classification. IEEE Journal of Biomedical and Health Informatics, vol. 19, no. 3, pp. 1118–1128, 2015. DOI: https://doi.org/10.1109/JBHI.2014.2335617.

    Google Scholar 

  37. Q. L. Li, B. W. Feng, L. P. Xie, P. Liang, H. S. Zhang, T. F. Wang. A cross-modality learning approach for vessel segmentation in retinal images. IEEE Transactions on Medical Imaging, vol. 35, no. 1, pp. 109–118, 2016. DOI: https://doi.org/10.1109/TMI.2015.2457891.

    Article  Google Scholar 

  38. M. Melinscak, P. Prentasic, S. Loncaric. Retinal vessel segmentation using deep neural networks. In Proceedings of the 10th International Conference on Computer Vision Theory and Applications, Berlin, Germany, pp. 577–582, 2015.

  39. P. Liskowski, K. Krawiec. Segmenting retinal blood vessels with deep neural networks. IEEE Transactions on Medical Imaging, vol. 35, no. 11, pp. 2369–2380, 2016. DOI: https://doi.org/10.1109/TMI.2016.2546227.

    Article  Google Scholar 

  40. J. T. Zhuang J. LadderNet: Multi-path networks based on U-Net for medical image segmentation. [Online], Available: https://arxiv.org/abs/1810.07810, 2018.

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, 230009, China

    Cong-Zhong Wu, Jun Sun, Liang-Feng Xu & Shu Zhan

  2. The Second Affiliated Hospital of Anhui Medical University, Hefei, 230601, China

    Jing Wang

Authors
  1. Cong-Zhong Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jun Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jing Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Liang-Feng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shu Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shu Zhan.

Additional information

Colored figures are available in the online version at https://link.springer.com/journal/11633

Cong-Zhong Wu received the M. Sc. degree in microwave specialty from Institute of Plasma Physics, Chinese Academy of Sciences, China in 1990. He is now an associate professor of School of Computer and Information Engineering, Hefei University of Technology, China.

His research interests include wireless communication, DSP technology and application, embedded application system development.

Jun Sun received the B.Sc. degree in electronic information engineering from Jiangsu University, China in 2018. Now, he is a graduate student in Hefei University of technology, China.

His research interests include deep learning and computer vision.

Jing Wang received the M. D. degree in ophthalmic optometry from Anhui Medical University, China in 2008. She is now an attending physician in the Second Affiliated Hospital of Anhui Medical University, China.

Her research interests include second laser treatment of myopia, high myopia comprehensive treatment, myopia and hyperopia correction.

Liang-Feng Xu received the B. Sc. degree in electronic information engineering from Hefei University of technology, China in 1993, and received the M.Eng. degree in signal and information processing from School of Computer and Information Technology, Hefei University of Technology, China in 2003. Since 1993, he has been engaged in teaching and scientific research in School of Computer and Information, Hefei University of Technology, China.

Her research interests include wireless communication, image processing and facial expression recognition.

Shu Zhan received Ph.D. degree in information and communication system from University of Science and Technology of China, China in 2000. He has been a postdoctoral researcher in School of Information, Tokyo University of Japan from 2002 to 2004. He has been a professor in School of Computer Science and Information Engineering, Hefei University of Technology, China since 1993.

His research interests include digital image analysis, medical image analysis and pattern recognition.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, CZ., Sun, J., Wang, J. et al. Encoding-decoding Network With Pyramid Self-attention Module For Retinal Vessel Segmentation. Int. J. Autom. Comput. 18, 973–980 (2021). https://doi.org/10.1007/s11633-020-1277-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11633-020-1277-0

Keywords

Search

Navigation