Abstract
We present a novel display and visual interaction paradigm, which aims at reducing the complexity of understanding the spatial transformations between the surgeon’s viewpoint, the patient, the pre- or intra-operative 2D and 3D data, and surgical tools during computer assisted interventions. To the best of our knowledge, this is the first work in which the traditional interventional display, for example in surgical navigation systems, is registered both to the patient and to the surgeon’s view. The closest concept is that of traditional Augmented Reality windows in which a semitransparent or video see-through display is positioned between the surgeon and the patient. In such cases, the system was providing an AR view into the patient. In the new concept introduced here, the surgeon keeps his/her own direct view to the patient without any need for additional display or direct view augmentation, but the monitor used in the operating room is now registered to the patient and surgeon’s viewpoint. The display could act as fixed viewing frustum or as a mirror frustum relative to the surgeon’s view. This allows the physicians to effortlessly relate their view of tools and patient to the virtual representation of the patient data. In this paper, the first realization and implementation of such a concept is presented and three clinical partners have tested the system and their first feedback is discussed in detail. They unanimously believe that this concept opens the path for facilitating interactive exploration of data and more intuitive navigation guidance in computer assisted interventions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We define \(^{A}T_{B}\) to represent a transformation from coordinate system \(A\) into \(B\).
References
Bichlmeier, C., Heining, S.M., Feuerstein, M., Navab, N.: The virtual mirror: a new interaction paradigm for augmented reality environments. IEEE Trans. Med. Imaging 28(9), 1498–1510 (2009). https://doi.org/10.1109/TMI.2009.2018622
Blackwell, M., Nikou, C., DiGioia, A.M., Kanade, T.: An image overlay system for medical data visualization. Med. Image Anal. 4(1), 67–72 (2000). https://doi.org/10.1016/S1361-8415(00)00007-4
Drouin, S., Collins, D., Kersten-Oertel, M.: Interaction in augmented reality image-guided surgery. In: Mixed and Augmented Reality in Medicine, pp. 99–114. CRC Press (2018). https://doi.org/10.1201/9781315157702-7
Eagleson, R., de Ribaupierre, S.: Visual perception and human-computer interaction in surgical augmented and virtual reality environments. In: Mixed and Augmented Reality in Medicine, pp. 83–98. CRC Press (2018). https://doi.org/10.1201/9781315157702-6
Goebbels, G., et al.: Development of an augmented reality system for intra-operative navigation in maxillo-facial surgery. In: Proceedings BMBF Statustagung, Stuttgart, Germany, pp. 237–246, January 2002
Houston, E.J.: The x-rays. In: Young, H.W. (ed.) Popular Electricity, vol. 2, pp. 4–9. Popular Electricity Publishing Company (1909)
Kooima, R.: Generalized perspective projection (2009). http://csc.lsu.edu/kooima/articles/genperspective
Mezger, U., Jendrewski, C., Bartels, M.: Navigation in surgery. Langenbeck’s Arch. Surg. 398(4), 501–514 (2013). https://doi.org/10.1007/s00423-013-1059-4
Navab, N., Blum, T., Wang, L., Okur, A., Wendler, T.: First deployments of augmented reality in operating rooms. Computer 45(7), 48–55 (2012). https://doi.org/10.1109/MC.2012.75
Navab, N., Traub, J., Sielhorst, T., Feuerstein, M., Bichlmeier, C.: Action-and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput. Graph. Appl. 27(5), 10–14 (2007). https://doi.org/10.1109/MCG.2007.117
Park, J.S., Chung, M.S., Hwang, S.B., Lee, Y.S., Har, D.H., Park, H.S.: Visible korean human: improved serially sectioned images of the entire body. IEEE Trans. Med. Imaging 24(3), 352–360 (2005). https://doi.org/10.1109/tmi.2004.842454
Schnaider, M., Schwald, B., Seibert, H., Weller, T.: Medarpa - a medical augmented reality system for minimal-invasive interventions. Stud. Health Technol. Inform. 94, 312–314 (2003). https://doi.org/10.3233/978-1-60750-938-7-312
Tuceryan, M., et al.: Calibration requirements and procedures for a monitor-based augmented reality system. IEEE Trans. Vis. Comput. Graph. 1(3), 255–273 (1995). https://doi.org/10.1109/2945.466720
Weber, S., Klein, M., Hein, A., Krueger, T., Lueth, T.C., Bier, J.: The navigated image viewer – evaluation in maxillofacial surgery. In: Ellis, Randy E., Peters, Terry M. (eds.) MICCAI 2003. LNCS, vol. 2878, pp. 762–769. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39899-8_93
Wilkinson, E.P., Shahidi, R., Wang, B., Martin, D.P., Adler, J.R., Steinberg, G.K.: Remote-rendered 3D CT angiography (3DCTA) as an intraoperative aid in cerebrovascular neurosurgery. Comput. Aided Surg. 4(5), 256–263 (1999). https://doi.org/10.3109/10929089909148178
Acknowledgements
This work was partially supported by Stryker Leibinger GmbH & Co. KG.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (avi 71966 KB)
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Winkler, A., Eck, U., Navab, N. (2020). Spatially-Aware Displays for Computer Assisted Interventions. In: Martel, A.L., et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2020. MICCAI 2020. Lecture Notes in Computer Science(), vol 12263. Springer, Cham. https://doi.org/10.1007/978-3-030-59716-0_43
Download citation
DOI: https://doi.org/10.1007/978-3-030-59716-0_43
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-59715-3
Online ISBN: 978-3-030-59716-0
eBook Packages: Computer ScienceComputer Science (R0)
