Skip to main content
SpringerLink
Log in

Calculation of ophthalmic diagnostic parameters on a single eye image based on deep neural network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

It is necessary to manually measure many parameters of eyes when an ophthalmologist diagnoses, which is time consuming, unsanitary, subjective and unrepeatable. Those manually achieved parameters often risk clinical trials in challenges on objectivity, resulting in unreliable clinical conclusions. We designed a two-phase algorithm to automatically measure these parameters instead of manual measurement to facilitate the diagnosis procedure of ptosis, eyelid retraction and other eye pathologies. Firstly, cornea, sclera, and internal and external canthus were identified using a multi-task convolutional neural network (CNN). Then we used the identification results to calculate a series of eye parameters needed in clinic. Totally 9 widely accepted parameters were calculated, including the height of ocular fissure, the longitudinal and transverse diameters of cornea, etc. The experimental results showed that most of parameters had achieved good accuracy, and most errors were less than 2 mm. We found that our approach can improve the measurements accuracy of eyelid, conjunctiva, cornea, sclera, and internal and external canthus, and promote efficiency and efficacy of relative clinical researches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Abate AF, Frucci M, Galdi C, Riccio D (2015) BIRD: watershed based iris detection for mobile devices. Pattern Recogn Lett 57:43–51. https://doi.org/10.1016/j.patrec.2014.10.017

    Article  Google Scholar 

  2. Badrinarayanan V, Kendall A, Cipolla R (2017) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. IEEE Trans Pattern Analysis and Machine Intelligence 39(12):2481–2495. https://doi.org/10.1109/tpami.2016.2644615

    Article  Google Scholar 

  3. Boles WW, Boashash B (1998) A human identification technique using images of the iris and wavelet transform. IEEE Trans Signal Processing 46(4):1185–1188. https://doi.org/10.1109/78.668573

    Article  Google Scholar 

  4. Castro A, Benito A, Manzanera S et al (2018) Three-dimensional cataract crystalline lens imaging with swept-source optical coherence tomography. Investigative Ophthalmology Visual Science 59:897–903. https://doi.org/10.1167/iovs.17-23596

    Article  Google Scholar 

  5. Chen C, Ross A (2018) A multi-task convolutional neural network for joint iris detection and presentation attack detection. IEEE Winter Applications of Computer Vision Workshops, pp:44–51. https://doi.org/10.1109/WACVW.2018.00011

  6. Chen LC, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans. Pattern Analysis and Machine Intelligence 40(4):834–848. https://doi.org/10.1109/TPAMI.2017.2699184

    Article  Google Scholar 

  7. Das A, Pal U, Ferrer MA et al (2016) SSRBC 2016: sclera segmentation and recognition benchmarking competition. Int. Conf. Biometrics, pp 1–6. https://doi.org/10.1109/ICB.2016.7550069

  8. Das S, De Ghosh I, Chattopadhyay A (2020) An efficient deep learning strategy: its application in sclera segmentation. IEEE Applied Signal Processing Conference, pp 232–236. https://doi.org/10.1109/ASPCON49795.2020.9276718

  9. Daugman JG (1993) High confidence visual recognition of persons by a test of statistical independence. IEEE Trans. Pattern Analysis and Machine Intelligence 15(11):1148–1161. https://doi.org/10.1109/34.244676

    Article  Google Scholar 

  10. Daugman J (2004) How iris recognition works. IEEE Trans Circuits and Systems for Video Technology 14(1):21–30. https://doi.org/10.1016/B978-0-12-374457-9.00025-1

    Article  Google Scholar 

  11. Daugman J (2007) New methods in iris recognition. IEEE trans. Systems Man Cybernetics, Part B (Cybernetics) 37(5):1167–1175. https://doi.org/10.1109/TSMCB.2007.903540

    Article  Google Scholar 

  12. De Marsico M, Nappi M, Riccio D et al (2015) Mobile iris challenge evaluation (miche)-i, biometric iris dataset and protocols. Pattern Recogn Lett 57:17–23. https://doi.org/10.1016/j.patrec.2015.02.009

    Article  Google Scholar 

  13. Gao X, Lin S, Wong TY (2015) Automatic feature learning to grade nuclear cataracts based on deep learning. IEEE trans. Biomed Eng 62:2693–2701. https://doi.org/10.1109/TBME.2015.2444389

    Article  Google Scholar 

  14. Gulshan V, Peng L, Coram M, Stumpe MC, Wu D, Narayanaswamy A, Venugopalan S, Widner K, Madams T, Cuadros J, Kim R, Raman R, Nelson PC, Mega JL, Webster DR (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. Jama 316(22):2402–2410. https://doi.org/10.1001/jama.2016.17216

    Article  Google Scholar 

  15. Jayanthi J, Lydia EL, Krishnaraj N, Jayasankar T, Babu RL, Suji RA (2020) An effective deep learning features based integrated framework for iris detection and recognition. Journal of ambient intelligence and humanized computing, pp 1-11. 12:3271–3281. https://doi.org/10.1007/s12652-020-02172-y

  16. Kerrigan D, Trokielewicz M, Czajka A et al (2019) Iris recognition with image segmentation employing retrained off-the-shelf deep neural networks. Int. Conf. Biometrics, pp 1–7. https://doi.org/10.1109/ICB45273.2019.8987299

  17. Li Y, Yuan L, Vasconcelos N (2019) Bidirectional learning for domain adaptation of semantic segmentation. Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp 6936–6945. https://doi.org/10.1109/CVPR.2019.00710

  18. Litjens G, Kooi T, Bejnordi BE, Setio AAA, Ciompi F, Ghafoorian M, van der Laak JAWM, van Ginneken B, Sánchez CI (2017) A survey on deep learning in medical image analysis. Med Image Anal 42(9):60–88. https://doi.org/10.1016/j.media.2017.07.005

    Article  Google Scholar 

  19. Liu W, Anguelov D, Erhan D et al (2016) SSD: single shot multibox detector. European Conf. Computer Vision, pp 21–37

  20. Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vis 128:261–318. https://doi.org/10.1007/s11263-019-01247-4

    Article  MATH  Google Scholar 

  21. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp 3431–3440

  22. Lucio DR, Laroca R, Severo E et al (2018) Fully connected networks and generative neural networks applied to sclera segmentation. Int. Conf. Biometrics Theory, Applications and Systems, Redondo Beach, CA, USA, pp. 1–7. https://doi.org/10.1109/BTAS.2018.8698597

  23. Mark J, Grinsven V, Ginneken BV et al (2016) Fast convolutional neural network training using selective data sampling: application to hemorrhage detection in colour fundus images. IEEE Trans Medical Imaging 35(5):1273–1284. https://doi.org/10.1109/TMI.2016.2526689

    Article  Google Scholar 

  24. Marku N, Frljak M, Pandi IS et al (2014) Eye pupil localization with an ensemble of randomized trees. Pattern Recogn 47(2):578–587. https://doi.org/10.1016/j.patcog.2013.08.008

    Article  Google Scholar 

  25. Radu P, Ferryman J, Wild P (2015) A robust sclera segmentation algorithm. Proc. IEEE Int. Conf. Biometrics Theory, Applications and Systems, Arlington, USA, pp. 1–6. https://doi.org/10.1109/BTAS.2015.7358746

  26. Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp 7263–7271

  27. Redmon J, Farhadi A (2018) Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767.

  28. Redmon J, Divvala S, Girshick R et al (2016) You only look once: unified, real-time object detection. IEEE Conf. Computer Vision and Pattern Recognition, LAS VEGAS, USA, pp. 779–788.

  29. Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. Int. Conf. Medical image computing and computer-assisted intervention, Munich, Germany, pp 234-241.https://doi.org/10.1007/978-3-319-24574-4_28

  30. Rot P, Vitek M, Grm K et al (2020) Deep sclera segmentation and recognition. Handbook of vascular biometrics. Springer, pp 395–432

  31. Setio A, Ciompi F, Litjens G et al (2016) Pulmonary nodule detection in CT images: false positive reduction using multi-view convolutional networks. IEEE Trans. Medical Imaging 35(5):1160–1169. https://doi.org/10.1109/TMI.2016.2536809

    Article  Google Scholar 

  32. Severo E, Laroca R, Bezerra C et al (2018) A benchmark for iris location and a deep learning detector evaluation. Int. Jo. Conf. Neural Networks, pp 1–7. https://doi.org/10.1109/IJCNN.2018.8489638

  33. Susitha N, Subban R (2019) Reliable pupil detection and iris segmentation algorithm based on SPS. Cogn Syst Res 57:78–84. https://doi.org/10.1016/j.cogsys.2018.09.029

    Article  Google Scholar 

  34. Thakker MM, Rubin PA (2002) Mechanisms of acquired blepharoptosis. Ophthalmol Clin N Am 15(1):101–111. https://doi.org/10.1016/s0896-1549(01)00005-0

    Article  Google Scholar 

  35. Tian Z, Shen C, Chen H et al (2019) FCOS: fully convolutional one-stage object detection. Proc. Int. Conf. Computer Vision, pp 9627–9636. https://doi.org/10.1109/ICCV.2019.00972

  36. Wang C, Zhu Y, Liu Y, et al (2019) Joint iris segmentation and localization using deep multi-task learning framework. arXiv preprint arXiv:1901.11195.

  37. Wang C , He Y , Liu Y , et al (2019) ScleraSegNet: an improved U-net model with attention for accurate sclera segmentation. The 12th IAPR Int. Conf. Biometrics, Crete, Greece, pp 1-8. https://doi.org/10.1109/ICB45273.2019.8987270

  38. Wood E, Baltrušaitis T, Morency LP et al (2016) Learning an appearance-based gaze estimator from one million synthesised images. Proc. Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, pp 131–138. https://doi.org/10.1145/2857491.2857492

  39. Yang P, Du B, Shan S et al (2004) A novel pupil localization method based on gaboreye model and radial symmetry operator. Int. Conf. Image Processing, Singapore 1:67–70. https://doi.org/10.1109/ICIP.2004.1418691

    Article  Google Scholar 

  40. Yuille AL, Hallinan PW, Cohen DS (1992) Feature extraction from faces using deformable templates. Int J Comput Vis 8(2):99–111. https://doi.org/10.1109/CVPR.1989.37836

    Article  Google Scholar 

  41. Zhou E, Fan H, Cao Z et al (2013) Extensive facial landmark localization with coarse-to-fine convolutional network Cascade. Proc. Int. Conf. Computer Vision Workshops, pp 386–391. https://doi.org/10.1109/ICCVW.2013.58

Download references

Author information

Author notes

  1. Xiangyang Zhu and Xuefei Song contributed equally to this work.

Authors and Affiliations

  1. Institute of Image Communication and Network Engineering, Shanghai Jiao Tong University, No. 800, Dong Chuan Road, Shanghai, China

    Xiangyang Zhu, Xiongkuo Min, Wei Sun, Jia Wang & Guangtao Zhai

  2. Shanghai Ninth People’s Hospital, Shanghai Jiao Tong University School of Medicine, No. 639, Zhi Zao Ju Road, Shanghai, China

    Xuefei Song & Huifang Zhou

Authors
  1. Xiangyang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xuefei Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiongkuo Min
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Huifang Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jia Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Guangtao Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiangyang Zhu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Song, X., Min, X. et al. Calculation of ophthalmic diagnostic parameters on a single eye image based on deep neural network. Multimed Tools Appl 81, 2311–2331 (2022). https://doi.org/10.1007/s11042-021-11047-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-11047-z

Keywords

Search

Navigation