Abstract
Swept-Source Optical Coherence Tomography (SS-OCT) integrated with surgical microscopes has enabled fast, high-resolution, and volumetric visualization of delicate tissue-instrument interactions. However, some visual features, which provide essential perceptual information in microscopic surgery, are not present in 4D OCT. Such a feature is the shadow of the surgical instruments cast onto the retina by the endo-illumination probe, which is among the most important cognitive cues for perceptual distance estimation. In this work, we propose Semantic Virtual Shadows (SVS), a novel concept to artificially generate instrument-specific shadows in OCT volumes, enabling naturally non-existent but important perceptual cues that are present in microscopic surgery. Semantic scene information is leveraged by considering only voxels associated with shadow-casting and shadow-receiving objects, identified using a learning-based approach and efficient volume processing, respectively. Real-time performance is achieved by a precomputed semantic shadow volume texture that assigns a shadowing factor to each voxel associated with a shadow-receiving object. The novelty of the method includes not only instrument-specific shadowing on the surface anatomy but also exclusively on deep-seated subsurface structures, providing advantages for various vitreoretinal procedures. Our user study indicates the benefits of the method for 4D OCT-guided surgery in several cognitive and performance-specific aspects.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bleicher, I.D., Jackson-Atogi, M., Viehland, C., Gabr, H., Izatt, J.A., Toth, C.A.: Depth-based, motion-stabilized colorization of microscope-integrated optical coherence tomography volumes for microscope-independent microsurgery. Transl. Vis. Sci. Technol. 7(6), 1–1 (2018)
Britten, A., et al.: Surgical microscope integrated MHz SS-OCT with live volumetric visualization. Biomed. Optics Express 14(2), 846–865 (2023)
Carrasco-Zevallos, O.M., Viehland, C., Keller, B., McNabb, R.P., Kuo, A.N., Izatt, J.A.: Constant linear velocity spiral scanning for near video rate 4D OCT ophthalmic and surgical imaging with isotropic transverse sampling. Biomed. Optics Express 9(10), 5052 (2018)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Jeong, H., Kim, H.J., Hyeon, M.G., Kim, P., Choi, Y., Kim, B.M.: Shadow extension for ray casting enhances volumetric visualization in real-time 4D-OCT. Optics Commun. 460, 125237 (2020)
Jo, Y.J., Heo, D.W., Shin, Y.I., Kim, J.Y.: Diurnal variation of retina thickness measured with time domain and spectral domain optical coherence tomography in healthy subjects. Invest. Ophthalmol. Vis. Sci. 52(9), 6497–6500 (2011)
Kolb, J.P., Draxinger, W., Klee, J., Pfeiffer, T., Eibl, M., Klein, T., Wieser, W., Huber, R.: Live video rate volumetric OCT imaging of the retina with multi-MHz A-scan rates. PLoS ONE 14(3), e0213144 (2019)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Roodaki, H., di San Filippo, C.A., Zapp, D., Navab, N., Eslami, A.: A surgical guidance system for big-bubble deep anterior lamellar keratoplasty. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9900, pp. 378–385. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46720-7_44
Ropinski, T., Kasten, J., Hinrichs, K.: Efficient shadows for GPU-based volume raycasting (2008)
Sommersperger, M., et al.: Surgical scene generation and adversarial networks for physics-based iOCT synthesis. Biomed. Optics Express 13(4), 2414–2430 (2022)
Trout, R.M., et al.: Methods for real-time feature-guided image fusion of intrasurgical volumetric optical coherence tomography with digital microscopy. Biomed. Optics Express 14(7), 3308–3326 (2023)
Viehland, C., et al.: Enhanced volumetric visualization for real time 4d intraoperative ophthalmic swept-source oct. Biomed. Optics Express 7(5), 1815–1829 (2016)
Weiss, J., Eck, U., Nasseri, M.A., Maier, M., Eslami, A., Navab, N.: Layer-aware iOCT volume rendering for retinal surgery. In: Kozlíková, B., Linsen, L., Vázquez, P.P., Lawonn, K., Raidou, R.G. (eds.) Eurographics Workshop on Visual Computing for Biology and Medicine. The Eurographics Association (2019)
Weiss, J., Sommersperger, M., Nasseri, A., Eslami, A., Eck, U., Navab, N.: Processing-aware real-time rendering for optimized tissue visualization in intraoperative 4D OCT. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12265, pp. 267–276. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59722-1_26
Wilkins, J.R., et al.: Characterization of epiretinal membranes using optical coherence tomography. Ophthalmology 103(12), 2142–2151 (1996)
Zhang, T., Kho, A.M., Yiu, G., Srinivasan, V.J.: Visible light optical coherence tomography (OCT) quantifies subcellular contributions to outer retinal band 4. Transl. Vis. Sci. Technol. 10(3), 30–30 (2021)
Acknowledgements
This work is partially supported and the data is provided by Carl Zeiss Meditec. The authors wish to thank SynthesEyes (https://syntheseyes.de) for providing the excellent simulation setup for the user study.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Sommersperger, M. et al. (2023). Semantic Virtual Shadows (SVS) for Improved Perception in 4D OCT Guided Surgery. In: Greenspan, H., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. MICCAI 2023. Lecture Notes in Computer Science, vol 14228. Springer, Cham. https://doi.org/10.1007/978-3-031-43996-4_39
Download citation
DOI: https://doi.org/10.1007/978-3-031-43996-4_39
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43995-7
Online ISBN: 978-3-031-43996-4
eBook Packages: Computer ScienceComputer Science (R0)
