Skip to main content
SpringerLink
Log in

E-Net: a novel deep learning framework integrating expert knowledge for glaucoma optic disc hemorrhage segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Glaucoma is a serious eye disease and glaucoma optic disc hemorrhage (GODH) is an important diagnostic indicator for glaucoma. Deep-learning-based medical image segmentation methods for automatic optic cup and disc segmentation have made tremendous progress. However, when it comes to the segmentation of GODH, classical deep learning technologies face two main challenges: the difficulties in distinguishing GODH from the end points or bending points of blood vessels, and the imbalance between the pixel classes of the target area and the background area. In this paper, we proposed a deep learning framework integrating expert knowledge (E-Net) for the segmentation of GODH in fundus images. This E-Net consisted of a primary network for GODH segmentation and two auxiliary networks for extraction of optic disc (OD) and blood vessels. The segmentation probability maps from the two auxiliary networks were used to improve the segmentation accuracy of GODH, via expert knowledge loss functions and attention mechanism. Moreover, we designed a weighted segmentation accuracy loss function to balance the segmentation accuracy of the target and background region, thus fully mining the substantial information in the fundus images. The proposed E-Net was verified on a GODH dataset from Beijing Tongren Hospital. The experiments showed that the proposed E-Net achieved state-of-the-art results on this dataset.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

All data related to the study were not published. Any request for data can be made to the corresponding author and is subject to ethics approval.

References

  1. Bengtsson B, Leske MC, Yang Z, Heijl A (2008) Disc hemorrhages and treatment in the early manifest glaucoma trial. Ophthalmology 115:2044–2048

    Article  Google Scholar 

  2. Cao H, Wang Y, Chen J, Jiang D, Zhang X, Tian Q, Wang M (2021) Swin-Unet: Unet-like pure transformer for medical image segmentation. Computer vision-ECCV 2022 workshops: Tel Aviv, Israel, October 23–27, 2022, Proceedings Part Ill (pp 205–218)

  3. Chaudhari S, Polatkan G, Ramanath R, Mithal V (2019) An attentive survey of attention models. arXiv:1904.02874

  4. Chen J, Lu Y, Yu Q, Luo X, Zhou Y (2021) TransUNet: Transformers Make Strong Encoders for Medical Image Segmentation. arXiv preprint arXiv: 2102.04306

  5. Chen H, Qin Z, Ding Y, Tian L, Qin Z (2020) Brain tumor segmentation with deep convolutional symmetric neural network. Neurocomputing 392:305–313

    Article  Google Scholar 

  6. Christopher M, Belghith A, Bowd C, Proudfoot JA, Goldbaum MH, Weinreb RN, Girkin CA, Liebmann JM, Zangwill LM (2018) Performance of deep learning architectures and transfer learning for detecting glaucomatous optic neuropathy in fundus photographs. Sci Rep-Uk 8:1–13

    Google Scholar 

  7. Eun KK, Ho PK (2017) Optic disc hemorrhage in glaucoma: pathophysiology and prognostic significance. Curr Opin Ophthalmol 28:105–112

    Article  Google Scholar 

  8. Evan S, Jonathan L, Trevor D (2017) Fully convolutional networks for semantic segmentation. Ieee T Pattern Anal 39:640–651

    Article  Google Scholar 

  9. Feng S, Zhuo Z, Pan D, Tian Q (2020) CcNet: A cross-connected convolutional network for segmenting retinal vessels using multi-scale features. Neurocomputing 392:268–276

    Article  Google Scholar 

  10. He J, Baxter SL, Xu J, Xu J, Zhou X, Zhang K (2019) The practical implementation of artificial intelligence technologies in medicine. Nat Med 25:30–36. https://doi.org/10.1038/s41591-018-0307-0

    Article  Google Scholar 

  11. Hood DC (2018) Efficacy of a deep learning system for detecting glaucomatous optic neuropathy based on color fundus photographs. Ophthalmology 125:1119–1206

    Article  Google Scholar 

  12. Hssayeni MD, Croock MS, Salman AD, Al-khafaji HF, Yahya ZA, Ghoraani B (2020) Intracranial hemorrhage segmentation using a deep convolutional model. Data 5

  13. Huang H, Lin L, Tong R, Hu H, Wu J (2020) UNet 3+: a full-scale connected UNet for medical image segmentation. ICASSP, pp 1055–1059

  14. Huazhu F, Jun C, Yanwu X, Kee WDW, Jiang L, Xiaochun C (2018) Joint optic disc and cup segmentation based on multi-label deep network and polar transformation. Ieee T Med Imaging 37:1597–1605

    Article  Google Scholar 

  15. Huazhu F, Jun C, Yanwu X, Changqing Z, Kee WDW, Jiang L, Xiaochun C (2018) Disc-aware ensemble network for glaucoma screening from fundus image. Ieee T Med Imaging 37:2493–2501

    Article  Google Scholar 

  16. Jiang Y, Tan N, Peng T, Zhang H (2019) Retinal vessels segmentation based on dilated multi-scale convolutional neural network. Ieee Access 7:76342–76352

    Article  Google Scholar 

  17. Jonas JB, Budde WM, Panda-Jonas S (1999) Ophthalmoscopic evaluation of the optic nerve head. Surv Ophthalmol 43:293–320

    Article  Google Scholar 

  18. Kamili A, Fatima I, Hassan M, Parah SA, Ambati LS (2020) Embedding information reversibly in medical images for e-health. J Intell Fuzzy Syst 39:8389–8398

    Article  Google Scholar 

  19. Keetha NV, Babu P, Annavarapu C (2020) U-Det: a modified U-Net architecture with bidirectional feature network for lung nodule segmentation. arXiv:2003.09293

  20. Krizhevsky A, Sutskever I, Hinton GE (2017) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90

    Article  Google Scholar 

  21. Liu H, Li L, Wormstone IM, Qiao C, Zhang C, Liu P, Li S, Wang H, Mou D, Pang R et al (2019) Development and validation of a deep learning system to detect glaucomatous optic neuropathy using fundus photographs. Jama Ophthalmol 137:1353–1360

    Article  Google Scholar 

  22. Liu L, Mai X, Hanruo L, Yang L, Xiaofei W, Lai J, Zulin W, Xiang F, Ningli W (2020) A large-scale database and a CNN model for attention-based glaucoma detection. Ieee T Med Imaging 39:413–424

    Article  Google Scholar 

  23. Lu S, Hu M, Li RR, Xu YL (2020) A novel adaptive weighted loss design in adversarial learning for retinal nerve fiber layer defect segmentation. Ieee Access 8:132348–132359

    Article  Google Scholar 

  24. Ran G, Guotai W, Tao S, Rui H, Michael A, Jan D, Sebastien O, Tom V, Shaoting Z (2020) CA-Net: comprehensive attention convolutional neural networks for explainable medical image segmentation. IEEE Trans Med Imaging 40:699–711

    Google Scholar 

  25. Ren S, He K, Ross G, Sun J (2017) Faster R-CNN: towards real-time object detection with region proposal networks. Ieee T Pattern Anal 39:1137–1149

    Article  Google Scholar 

  26. Ronneberger O, Fischer P, Brox T (2015) U-Net: convolutional networks for biomedical image segmentation. Medical Image Computing and Computer-Assisted Intervention-MICCA/ 2015: 18th international Conference, Munich, Germany, October 5-9, 2015, Proceedings, Part Ill 18(pp 234-241). Springer International Publishing

  27. Ross G, Jeff D, Trevor D, Jitendra M (2016) Region-based convolutional networks for accurate object detection and segmentation. Ieee T Pattern Anal 38:142–158

    Article  Google Scholar 

  28. Sevastopolsky A (2017) Optic disc and cup segmentation methods for glaucoma detection with modification of U-Net convolutional neural network. Pattern Recognit Image Anal 27:1597–1605

    Article  Google Scholar 

  29. Staal J, Abramoff MD, Niemeijer M, Viergever MA, van Ginneken B (2004) Ridge-based vessel segmentation in color images of the retina. Ieee T Med Imaging 23:501–509

    Article  Google Scholar 

  30. Tham YC, Li X, Wong TY, Quigley HA, Aung T, Cheng CY (2014) Global prevalence of glaucoma and projections of glaucoma burden through 2040: a systematic review and meta-analysis. Ophthalmology 121:2081–2090

    Article  Google Scholar 

  31. Ting DSW, Cheung CYL, Lim G, Tan GSW, Quang ND, Gan A, Hamzah H, Garcia-Franco R, Yeo IYS, Lee SY et al (2017) Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA 318:2211–2223

    Article  Google Scholar 

  32. Wang Z, Zou C, Cai W (2020) Small sample classification of hyperspectral remote sensing images based on sequential joint deeping learning model. Ieee Access 8:71353–71363

    Article  Google Scholar 

  33. Watanabe R, Muramatsu C, Ishida K, Sawada A, Hatanaka Y, Yamamoto T, Fujita H (2017) Automated detection of nerve fiber layer defects on retinal fundus images using fully convolutional network for early diagnosis of glaucoma. Medical Imaging 10134:826–832

    Google Scholar 

  34. Xu K, Ba J, Kiros R, Cho K, Courville A, Salakhutdinov R, Zemel R, Bengio Y (2015) Show, attend and tell: neural image caption generation with visual attention. International conference on machine learning, pp 2048–2057

  35. Xu H, Xie H, Liu Y, Cheng C, Zhang Y (2019) Deep cascaded attention network for multi-task brain tumor segmentation. Medical Image Computing and Computer Assisted Intervention – MICCAI 2019, pp 420–428

  36. Xu Y, Lu S, Li H, Li R (2019) Mixed maximum loss design for optic disc and optic cup segmentation with deep learning from imbalanced samples. Sensors (Basel, Switzerland) 19:4401

    Article  Google Scholar 

  37. Yu Y, Choi J, Kim Y, Yoo K, Lee SH, Kim G (2017) Supervising neural attention models for video captioning by human gaze Data IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 490–498

  38. Zhang Z, Fu H, Dai H, Shen J, Shao L (2019) ET-Net: a generic Edge-aTtention guidance network for medical image segmentation. MICCAI, pp 442–450

  39. Zhou Z, Siddiquee M, Tajbakhsh N, Liang J (2018) UNet++: a nested U-Net architecture for medical image segmentation. Deep Learning in Medical Image Analysis (DLMIA) Workshop, pp 3–11

  40. Zhu Y, Zhao C, Guo H, Wang J, Zhao X, Lu H (2018) Attention CoupleNet: fully convolutional attention coupling network for object detection. IEEE Trans Image Process 28:113–126

    Article  MathSciNet  Google Scholar 

  41. Zhu H, Yan X, Ng HT, Chang Y, Yuan FX (2020) Moving object detection with deep CNNs. Ieee Access 8:29729–29741

    Article  Google Scholar 

  42. Zilly J, Buhmann JM, Mahapatra D (2017) Glaucoma detection using entropy sampling and ensemble learning for automatic optic cup and disc segmentation. Comput Med Imag Grap 55:28–41

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China (U1830107), the Joint Project of Biomedical Translational Engineering Research Center of BUCT-CJFH (XK2022-02) and the National Science and Technology Major Project (Nos. 2019-I-0001-0001 and 2019-I-0019-0018).

Author information

Authors and Affiliations

  1. Department of Mathematics, Beijing University of Chemical Technology, Beijing, 100029, China

    Yongli Xu, Fan Meng & Hao Yang

  2. Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, 100081, China

    Shuai Lu

  3. School of Mathematics Sciences, Beihang University, Beijing, 100191, China

    Haihui Wang

  4. Department of Ophthalmology, Beijing Children’s Hospital, Capital Medical University, Beijing, 100045, China

    Man Hu

  5. Beijing Tongren Eye Center, Beijing Tongren Hospital, Capital Medical University, Beijing, 100045, China

    Man Hu

Authors
  1. Yongli Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fan Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shuai Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Haihui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Man Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Haihui Wang or Man Hu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xu, Y., Meng, F., Yang, H. et al. E-Net: a novel deep learning framework integrating expert knowledge for glaucoma optic disc hemorrhage segmentation. Multimed Tools Appl 82, 41207–41224 (2023). https://doi.org/10.1007/s11042-023-15174-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-15174-7

Keywords

Search

Navigation