Skip to main content

Ocular Stethoscope: Auditory Support for Retinal Membrane Peeling

  • Conference paper
  • First Online:
Medical Image Computing and Computer Assisted Intervention – MICCAI 2024 (MICCAI 2024)

Abstract

The peeling of an epiretinal membrane (ERM) is a complex procedure wherein a membrane, only a few micrometers thick, that develops on the retinal surface is delicately removed using microsurgical forceps. Insights regarding small gaps between the ERM and the retinal tissue are valuable for surgical decision-making, particularly in determining a suitable location to initiate the peeling. Depth-resolved imaging of the retina provided by intraoperative Optical Coherence Tomography (iOCT) enables visualization of this gap and supports decision-making. The common presentation of iOCT images during surgery in juxtaposition with the microscope view however requires surgeons to move their gaze from the surgical site, affecting proprioception and cognitive load.

In this work, we introduce an alternative method utilizing auditory feedback as a sensory channel, designed to intuitively enhance the perception of ERM elevations. Our approach establishes an unsupervised innovative mapping between real-time OCT A-scans and the parameters of an acoustic model. This acoustic model conveys the physical characteristics of tissue structure through distinctive sound textures, at microtemporal resolution. Our experiments show that even subtle ERM elevations can be sonified. Expert clinician feedback confirms the impact of our method and an initial user study with 15 participants demonstrates the potential to perceive the gap between the ERM and the retinal tissue exclusively through auditory cues.

S. Matinfar, S. Dehghani and M. Sommersperger—Contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    https://github.com/mi-creative/miPhysics_Processing.

  2. 2.

    https://www.syntheseyes.de/.

References

  1. Adrien, J.M.: The missing link: Modal synthesis. In: Representations of musical signals, pp. 269–298 (1991)

    Google Scholar 

  2. Ahmad, A., Adie, S.G., Wang, M., Boppart, S.A.: Sonification of optical coherence tomography data and images. Optics Express 18(10), 9934–9944 (2010)

    Article  Google Scholar 

  3. Black, D., Hansen, C., Nabavi, A., Kikinis, R., Hahn, H.: A survey of auditory display in image-guided interventions. International journal of computer assisted radiology and surgery 12, 1665–1676 (2017)

    Article  Google Scholar 

  4. Britten, A., Matten, P., Weiss, J., Niederleithner, M., Roodaki, H., Sorg, B., Hecker-Denschlag, N., Drexler, W., Leitgeb, R.A., Schmoll, T.: Surgical microscope integrated mhz ss-oct with live volumetric visualization. Biomedical Optics Express 14(2), 846–865 (2023)

    Article  Google Scholar 

  5. Cereda, M.G., Parrulli, S., Douven, Y., Faridpooya, K., van Romunde, S., Hüttmann, G., Eixmann, T., Schulz-Hildebrandt, H., Kronreif, G., Beelen, M., et al.: Clinical evaluation of an instrument-integrated oct-based distance sensor for robotic vitreoretinal surgery. Ophthalmology Science 1(4), 100085 (2021)

    Article  Google Scholar 

  6. Cook, P.R.: Physically informed sonic modeling (phism): Percussive synthesis. In: Proceedings of the 1996 International Computer Music Conference. pp. 228–231. The International Computer Music Association (1996)

    Google Scholar 

  7. Díaz-Valverde, A., Wu, L.: To peel or not to peel the internal limiting membrane in idiopathic epiretinal membranes. Retina 38, S5–S11 (2018)

    Article  Google Scholar 

  8. Ehlers, J.P., Modi, Y.S., Pecen, P.E., Goshe, J., Dupps, W.J., Rachitskaya, A., Sharma, S., Yuan, A., Singh, R., Kaiser, P.K., et al.: The discover study 3-year results: feasibility and usefulness of microscope-integrated intraoperative oct during ophthalmic surgery. Ophthalmology 125(7), 1014–1027 (2018)

    Article  Google Scholar 

  9. Franinovic, K., Serafin, S.: Sonic interaction design. Mit Press (2013)

    Google Scholar 

  10. Gholami, P., Roy, P., Parthasarathy, M.K., Lakshminarayanan, V.: Octid: Optical coherence tomography image database. Computers & Electrical Engineering (2020)

    Google Scholar 

  11. Gionfrida, L., Roginska, A.: A novel sonification approach to support the diagnosis of alzheimer’s dementia. Frontiers in neurology 8,  647 (2017)

    Article  Google Scholar 

  12. Hansen, C., Black, D., Lange, C., Rieber, F., Lamadé, W., Donati, M., Oldhafer, K.J., Hahn, H.K.: Auditory support for resection guidance in navigated liver surgery. The International Journal of Medical Robotics and Computer Assisted Surgery 9(1), 36–43 (2013)

    Google Scholar 

  13. Hermann, T.: Taxonomy and definitions for sonification and auditory display (2008)

    Google Scholar 

  14. Hermann, T., Hunt, A., Neuhoff, J.G., et al.: The sonification handbook, vol. 1. Logos Verlag Berlin (2011)

    Google Scholar 

  15. Hermann, T., Ritter, H.: Listen to your data: Model-based sonification for data analysis. Advances in intelligent computing and multimedia systems (1999)

    Google Scholar 

  16. Laina, I., Rieke, N., Rupprecht, C., Vizcaíno, J.P., Eslami, A., Tombari, F., Navab, N.: Concurrent segmentation and localization for tracking of surgical instruments. In: Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017. pp. 664–672. Springer (2017)

    Google Scholar 

  17. Matinfar, S., Nasseri, M.A., Eck, U., Roodaki, H., Navab, N., Lohmann, C.P., Maier, M., Navab, N.: Surgical soundtracks: Towards automatic musical augmentation of surgical procedures. In: Medical Image Computing and Computer-Assisted Intervention- MICCAI 2017. pp. 673–681. Springer (2017)

    Google Scholar 

  18. Matinfar, S., Salehi, M., Dehghani, S., Navab, N.: From tissue to sound: Model-based sonification of medical imaging. In: International Conference on Medical Image Computing and Computer-Assisted Intervention. pp. 207–216. Springer (2023)

    Google Scholar 

  19. Matinfar, S., Salehi, M., Suter, D., Seibold, M., Dehghani, S., Navab, N., Wanivenhaus, F., Fürnstahl, P., Farshad, M., Navab, N.: Sonification as a reliable alternative to conventional visual surgical navigation. Scientific Reports (2023)

    Google Scholar 

  20. Pavlidis, M., Georgalas, I., Körber, N., et al.: Determination of a new parameter, elevated epiretinal membrane, by en face oct as a prognostic factor for pars plana vitrectomy and safer epiretinal membrane peeling. Journal of Ophthalmology (2015)

    Google Scholar 

  21. Romano, M.R., Ilardi, G., Ferrara, M., Cennamo, G., Parolini, B., Mariotti, C., Staibano, S., Cennamo, G.: Macular peeling-induced retinal damage: clinical and histopathological evaluation after using different dyes. Graefe’s Archive for Clinical and Experimental Ophthalmology 256, 1573–1580 (2018)

    Article  Google Scholar 

  22. Roodaki, H., Filippatos, K., Eslami, A., Navab, N.: Introducing augmented reality to optical coherence tomography in ophthalmic microsurgery. In: 2015 IEEE international symposium on mixed and augmented reality. pp. 1–6. IEEE (2015)

    Google Scholar 

  23. Roodaki, H., Navab, N., Eslami, A., Stapleton, C., Navab, N.: Sonifeye: Sonification of visual information using physical modeling sound synthesis. IEEE transactions on Visualization and Computer Graphics 23(11), 2366–2371 (2017)

    Article  Google Scholar 

  24. Schütz, L., Weber, E., Niu, W., Daniel, B., McNab, J., Navab, N., Leuze, C.: Audiovisual augmentation for coil positioning in transcranial magnetic stimulation. Computer Methods in Biomechanics and Biomedical Engineering: Imaging & Visualization 11(4), 1158–1165 (2023)

    Google Scholar 

  25. Seidel, G., Weger, M., Stadlmüller, L., Pichler, T., Haas, A.: Association of preoperative optical coherence tomography markers with residual inner limiting membrane in epiretinal membrane peeling. PLoS One 8(6), e66217 (2013)

    Article  Google Scholar 

  26. Sommersperger, M., Dehghani, S., Matten, P., Mach, K., Roodaki, H., Eck, U., Navab, N.: Intelligent virtual b-scan mirror (ivbm). In: International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer (2023)

    Google Scholar 

  27. Theunissen, F.E., Elie, J.E.: Neural processing of natural sounds. Nature Reviews Neuroscience 15(6), 355–366 (2014)

    Article  Google Scholar 

  28. Vermeer, K.A., Mo, J., Weda, J.J., Lemij, H.G., de Boer, J.F.: Depth-resolved model-based reconstruction of attenuation coefficients in optical coherence tomography. Biomedical optics express 5(1), 322–337 (2014)

    Article  Google Scholar 

  29. Weiss, J., Eck, U., Nasseri, M.A., Maier, M., Eslami, A., Navab, N.: Layer-aware ioct volume rendering for retinal surgery. In: VCBM. pp. 123–127 (2019)

    Google Scholar 

  30. Wilkins, J.R., Puliafito, C.A., Hee, M.R., Duker, J.S., Reichel, E., Coker, J.G., Schuman, J.S., Swanson, E.A., Fujimoto, J.G.: Characterization of epiretinal membranes using optical coherence tomography. Ophthalmology 103(12) (1996)

    Google Scholar 

  31. Yang, J., Barde, A., Billinghurst, M.: Audio augmented reality: a systematic review of technologies, applications, and future research directions. journal of the audio engineering society 70(10), 788–809 (2022)

    Google Scholar 

  32. Ziemer, T.: Three-dimensional sonification as a surgical guidance tool. Journal on Multimodal User Interfaces 17(4), 253–262 (2023)

    Article  Google Scholar 

  33. Ziemer, T., Schultheis, H., Black, D., Kikinis, R.: Psychoacoustical interactive sonification for short range navigation. Acta Acustica united with Acustica (2018)

    Google Scholar 

Download references

Acknowledgments

The authors thank SynthesEyes for providing an excellent simulation setup for the user study. The authors wish to acknowledge the partial support of Bavarian Research Foundation (BFS). Since the submission of this paper, Shervin Dehghani and Michael Sommersperger have been supported by BFS project under grant number AZ 1569-22.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Sasan Matinfar .

Editor information

Editors and Affiliations

Ethics declarations

Disclosure of Interests

The authors have no competing interests to declare that are relevant to the content of this article.

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Matinfar, S., Dehghani, S., Sommersperger, M., Faridpooya, K., Fairhurst, M., Navab, N. (2024). Ocular Stethoscope: Auditory Support for Retinal Membrane Peeling. In: Linguraru, M.G., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2024. MICCAI 2024. Lecture Notes in Computer Science, vol 15006. Springer, Cham. https://doi.org/10.1007/978-3-031-72089-5_41

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-72089-5_41

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-72088-8

  • Online ISBN: 978-3-031-72089-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics