Skip to main content
SpringerLink
Log in

Self-supervised patient-specific features learning for OCT image classification

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Deep learning’s great success in image classification is heavily reliant on large-scale annotated datasets. However, obtaining labels for optical coherence tomography (OCT) data requires the significant effort of professional ophthalmologists, which hinders the application of deep learning in OCT image classification. In this paper, we propose a self-supervised patient-specific features learning (SSPSF) method to reduce the amount of data required for well OCT image classification results. Specifically, the SSPSF consists of a self-supervised learning phase and a downstream OCT image classification learning phase. The self-supervised learning phase contains two self-supervised patient-specific features learning tasks. One is to learn to discriminate an OCT scan which belongs to a specific patient. The other task is to learn the invariant features related to patients. In addition, our proposed self-supervised learning model can learn inherent representations from the OCT images without any manual labels, which provides well initialization parameters for the downstream OCT image classification model. The proposed SSPSF achieves classification accuracy of 97.74% and 98.94% on the public RETOUCH dataset and AI Challenger dataset, respectively. The experimental results on two public OCT datasets show the effectiveness of the proposed method compared with other well-known OCT image classification methods with less annotated data.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Michelessi M, Lucenteforte E, Oddone F, Brazzelli M, Parravano M, Franchi S, Ng SM, Virgili G (2015) Optic nerve head and fibre layer imaging for diagnosing glaucoma. The Cochrane Database of Systematic Reviews 30(11):CD008803

    Google Scholar 

  2. Drexler W, Morgner U, Kärtner FX, Pitris C, Boppart SA, XD L., Ippen EP, Fujimoto JG (1999) In vivo ultrahigh-resolution optical coherence tomography. Opt Lett 24(17):1221–1223

    Article  CAS  PubMed  Google Scholar 

  3. Aumann S, Donner S, Fischer J, Tomography FM (2019) Optical coherence (OCT): principle and technical realization. Springer International Publishing, Cham, pp 59–85

    Google Scholar 

  4. Fang L, Li S, Cunefare D, Farsiu S (2017) Segmentation based sparse reconstruction of optical coherence tomography images. IEEE Trans Med Imaging 36(2):407–421

    Article  PubMed  Google Scholar 

  5. Guo A, Fang L, Qi M, Li S (2021) Unsupervised denoising of optical coherence tomography images with nonlocal-generative adversarial network. IEEE Trans Instrum Meas 70:1–12

    CAS  Google Scholar 

  6. Kafieh R, Rabbani H, Abramoff MD, Sonka M (2013) Curvature correction of retinal octs using graph-based geometry detection. Phys Med Biol 58(9):2925–2938

    Article  PubMed  PubMed Central  Google Scholar 

  7. Srinivasan PP, Kim LA, Mettu PS, Cousins SW, Comer GM, Izatt JA, Farsiu S (2014) Fully automated detection of diabetic macular edema and dry age-related macular degeneration from optical coherence tomography images. Biomedical Optics Express 5(10):3568–3577

    Article  PubMed  PubMed Central  Google Scholar 

  8. Venhuizen FG, van Ginneken B, van Asten F, van Grinsven MJJP, Fauser S, Hoyng CB, Theelen T, Sánchez CI (2017) Automated staging of age-related macular degeneration using optical coherence tomography. Investigative ophthalmology & visual science 58(4):2318–2328

    Article  Google Scholar 

  9. Soh L-K, Tsatsoulis C (1999) Texture analysis of sar sea ice imagery using gray level co-occurrence matrices. IEEE Transactions on geoscience and remote sensing 37(2):780–795

    Article  Google Scholar 

  10. Lemaître G, Rastgoo M, Massich J, Cheung CY, Wong TY, Lamoureux E, Milea D, Mériaudeau F., Sidibé D (2016) Classification of sd-oct volumes using local binary patterns: Experimental validation for dme detection. J Ophthalmol 2016:3298606–3298619

    PubMed  PubMed Central  Google Scholar 

  11. Albarrak A, Coenen F, Zheng Y et al (2013) Age-related macular degeneration identification in volumetric optical coherence tomography using decomposition and local feature extraction. In: Proceedings of 2013 international conference on medical image, understanding and analysis, pp 59–64

  12. Liu Y-Y, Chen M, Ishikawa H, Wollstein G, Schuman JS, Rehg JM (2011) Automated macular pathology diagnosis in retinal oct images using multi-scale spatial pyramid and local binary patterns in texture and shape encoding. Med Image Anal 15(5):748–759

    Article  PubMed  PubMed Central  Google Scholar 

  13. Venhuizen FG, van Ginneken B, Bloemen B, van Grinsven MJJP, Philipsen R, Hoyng C, Theelen T, Sánchez CI (2015) Automated age-related macular degeneration classification in oct using unsupervised feature learning. In: Medical imaging 2015: Computer-aided diagnosis, vol 9414. International Society for Optics and Photonics, pp 94141I1–94141I7

  14. Lee CS, Baughman DM, Lee AY (2017) Deep learning is effective for classifying normal versus age-related macular degeneration oct images. Ophthalmology Retina 1(4):322–327

    Article  PubMed  PubMed Central  Google Scholar 

  15. Rasti R, Rabbani H, Mehridehnavi A, Hajizadeh F (2017) Macular oct classification using a multi-scale convolutional neural network ensemble. IEEE Trans Med Imaging 37(4):1024–1034

    Article  Google Scholar 

  16. De Fauw J, Ledsam JR, Romera-Paredes B, Nikolov S, Tomasev N, Blackwell S, Askham H, Glorot X, O’Donoghue B, Visentin D et al (2018) Clinically applicable deep learning for diagnosis and referral in retinal disease. Nat Med 24(9):1342–1350

    Article  PubMed  Google Scholar 

  17. Rong Y, Xiang D, Zhu W, Yu K, Shi F, Fan Z, Chen X (2018) Surrogate-assisted retinal oct image classification based on convolutional neural networks. IEEE Journal of Biomedical and Health Informatics 23(1):253–263

    Article  PubMed  Google Scholar 

  18. Fang L, Wang C, Li S, Rabbani H, Chen X, lesion ZL (2019) Attention to Lesion-aware convolutional neural network for retinal optical coherence tomography image classification. IEEE Trans Med Imaging 38(8):1959–1970

    Article  PubMed  Google Scholar 

  19. Huang L, He X, Fang L, Rabbani H, Chen X (2019) Automatic classification of retinal optical coherence tomography images with layer guided convolutional neural network. IEEE Signal Processing Letters 26(7):1026–1030

    Article  Google Scholar 

  20. Roy AG, Conjeti S, Karri SPK, Sheet D, Katouzian A, Wachinger C, Navab N (2017) Relaynet: retinal layer and fluid segmentation of macular optical coherence tomography using fully convolutional networks. Biomedical Optics Express 8(8):3627–3642

    Article  PubMed  PubMed Central  Google Scholar 

  21. Apostolopoulos S, De Zanet S, Ciller C, Wolf S, Sznitman R (2017) Pathological oct retinal layer segmentation using branch residual u-shape networks. In: International conference on medical image computing and computer-assisted intervention. Springer, pp 294–301

  22. Ngo L, Cha J, Han J-H (2019) Deep neural network regression for automated retinal layer segmentation in optical coherence tomography images. IEEE Trans Image Process 29:303–312

    Article  Google Scholar 

  23. Bogunović H, Venhuizen F, Klimscha S, Apostolopoulos S, Bab-Hadiashar A, Bagci U, Beg MF, Bekalo L, Chen Q, Ciller C et al (2019) Retouch: the retinal oct fluid detection and segmentation benchmark and challenge. IEEE Trans Med Imaging 38(8):1858–1874

    Article  PubMed  Google Scholar 

  24. Lee J, Gharaibeh Y, Kolluru C, Zimin VN, Dallan LAP, Kim JN, Bezerra HG, Wilson DL (2020) Segmentation of coronary calcified plaque in intravascular oct images using a two-step deep learning approach. IEEE Access 8:225581–225593

    Article  PubMed  PubMed Central  Google Scholar 

  25. Xiao Y, Gao S, Chai Z, Zhou K, Zhang T, Zhao Y, Cheng J, Liu J (2020) Open-set oct image recognition with synthetic learning. In: 2020 IEEE 17Th international symposium on biomedical imaging (ISBI), pp 1788–1792

  26. Zang P, Gao L, Hormel TT, Wang J, You Q, Hwang TS, Dcardnet YJ (2021) Diabetic retinopathy classification at multiple levels based on structural and angiographic optical coherence tomography. IEEE Trans Biomed Eng 68(6):1859–1870

    Article  PubMed  PubMed Central  Google Scholar 

  27. He X, Deng Y, Fang L, Peng Q (2021) Multi-modal retinal image classification with modality-specific attention network. IEEE Trans Med Imaging 40(6):1591–1602

    Article  PubMed  Google Scholar 

  28. Elsawy A, Abdel-Mottaleb M (2021) A novel network with parallel resolution encoders for the diagnosis of corneal diseases. IEEE Trans Biomed Eng 1–1

  29. Athar S, Vahadane A, Joshi A, Dastidar TR (2018) Weakly supervised fluid filled region localization in retinal oct scans. In: 2018 IEEE 15Th international symposium on biomedical imaging (ISBI), pp 1467–1470

  30. Ma X, Ji Z, Niu S, Leng T, Rubin DL, Ms-cam QC (2020) Multi-scale class activation maps for weakly-supervised segmentation of geographic atrophy lesions in sd-oct images. IEEE Journal of Biomedical and Health Informatics 24(12):3443–3455

    Article  PubMed  Google Scholar 

  31. Yang J, Hu Y, Fang L, Cheng J, Liu J (2020) Universal digital filtering for denoising volumetric retinal oct and oct angiography in 3d shearlet domain. Opt Lett 45(3):694–697

    Article  PubMed  Google Scholar 

  32. Seeböck P, Waldstein SM, Klimscha S, Bogunovic H, Schlegl T, Gerendas BS, Donner R, Schmidt-Erfurth U, Langs G (2019) Unsupervised identification of disease marker candidates in retinal oct imaging data. IEEE Trans Med Imaging 38(4):1037–1047

    Article  PubMed  Google Scholar 

  33. Yang S, Zhou X, Wang J, Xie G, Lv C, Gao P, Lv B (2020) Unsupervised domain adaptation for cross-device oct lesion detection via learning adaptive features. In: 2020 IEEE 17Th international symposium on biomedical imaging (ISBI), pp 1570–1573

  34. Noroozi M, Favaro P (2016) Unsupervised learning of visual representations by solving jigsaw puzzles. In: European conference on computer vision. Springer, pp 69–84

  35. Pathak D, Krahenbuhl P, Donahue J, Darrell T, Efros AA (2016) Context encoders: Feature learning by inpainting. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2536–2544

  36. Sermanet P, Lynch C, Chebotar Y, Hsu J, Jang E, Schaal S, Levine S, Brain G (2018) Time-contrastive networks: Self-supervised learning from video. In: 2018 IEEE International conference on robotics and automation (ICRA). IEEE, pp 1134–1141

  37. Misra I, Zitnick CL, Hebert M (2016) Shuffle and learn: unsupervised learning using temporal order verification. In: European conference on computer vision. Springer, pp 527–544

  38. Devon Hjelm R, Fedorov A, Lavoie-Marchildon S, Grewal K, Bachman P, Trischler A, Bengio Y (2018) Learning deep representations by mutual information estimation and maximization. In: International conference on learning representations

  39. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: ICML 2020: 37Th international conference on machine learning, vol 1, pp 1597–1607

  40. Budenz DL, Anderson DR, Varma R, Schuman J, Cantor L, Savell J, Greenfield DS, Patella VM, Quigley HA, Tielsch J (2007) Determinants of normal retinal nerve fiber layer thickness measured by stratus oct. Ophthalmology 114(6):1046–1052

    Article  PubMed  Google Scholar 

  41. Wang Q, Wei WB, Wang YX, Yan YN, Yang JY, Zhou WJ, Chan SY, Xu L, Jonas JB (2020) Thickness of individual layers at the macula and associated factors: The beijing eye study 2011. BMC Ophthalmol 20(1):1–11

    Article  Google Scholar 

  42. Alamouti B, Funk J (2003) Retinal thickness decreases with age: an oct study. Br J Ophthalmol 87(7):899–901

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  43. Ahmed SMMS, Abdallah NH, Eliwa LS, Bayomy AM (2020) The influence of corneal astigmatism on retinal nerve fiber layer thickness and optic nerve head parameter measurements by optical coherence tomography. QJM: An International Journal of Medicine 113(Supplement_1):hcaa058–005

    Article  Google Scholar 

  44. Prakasam RK, Matuszewska-Iwanicka A, Fischer D-C, Schumann H, Tschöpe D., Stratmann B, Hettlich H-J, Guthoff RF, Stachs O, Röhlig M (2020) Thickness of intraretinal layers in patients with type 2 diabetes mellitus depending on a concomitant diabetic neuropathy Results of a cross-sectional study using deviation maps for oct data analysis. Biomedicines 8(7):190

    Article  CAS  PubMed Central  Google Scholar 

  45. Garcia-Medina JJ, del Rio-Vellosillo M, Palazon-Cabanes A, Pinazo-Duran MD, Zanon-Moreno V, maculopathy MPV-P (2020) Glaucomatous Thickness differences on inner and outer macular layers between ocular hypertension and early primary open-angle glaucoma using 8× 8 posterior pole algorithm of sd-oct. Journal of Clinical Medicine 9(5):1503

    Article  PubMed Central  Google Scholar 

  46. Belghazi MI, Baratin A, Rajeshwar S, Ozair S, Bengio Y, Courville A, Hjelm D (2018) Mutual information neural estimation. In: International conference on machine learning. PMLR, pp 531–540

  47. Gutmann M, estimation AH (2010) Noise-contrastive a new estimation principle for unnormalized statistical models. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics. JMLR Workshop and Conference Proceedings, pp 297–304

  48. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  49. Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: ICLR 2015 : International Conference on learning representations 2015

  50. Gidaris S, Singh P, Komodakis N (2018) Unsupervised representation learning by predicting image rotations. In: International conference on learning representations

  51. Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR, pp 1597–1607

  52. Wang J, Yi Y, Mao J, Huang Z, Huang C, Cnn-rnn WX (2016) A unified framework for multi-label image classification. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR), pp 2285–2294

Download references

Author information

Authors and Affiliations

  1. The College of Electrical and Information Engineering, Hunan University, Changsha, Hunan, China

    Leyuan Fang, Jiahuan Guo & Xingxin He

  2. The College of Engineering and Computer Science, The Australian National University, Canberra, ACT 2601, Australia

    Muxing Li

Authors
  1. Leyuan Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jiahuan Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xingxin He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Muxing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xingxin He.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, L., Guo, J., He, X. et al. Self-supervised patient-specific features learning for OCT image classification. Med Biol Eng Comput 60, 2851–2863 (2022). https://doi.org/10.1007/s11517-022-02627-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02627-8

Keywords

Search

Navigation