Abstract
In recent years, many marker-based augmented reality (AR) surgical navigation systems have been developed to assist with surgeries. However, the current marker-based registration methods encounter challenges related to complex marker placement and adjustments, limiting their widespread adoption in clinical settings, especially in diverse surgical scenarios. This study presents a novel marker-based multi-camera AR surgical navigation system that encompasses a robust marker detection framework and a high-precision registration system. Our marker-based registration system addresses the challenges of marker placement by utilizing an extended perception range and a high-accuracy calibration solution. Moreover, we introduce an innovative deep learning-based framework designed to provide accurate and robust marker detection even under challenging lighting and motion conditions. Validation experiments underscore the superiority of our system over standard methods, exhibiting enhanced performance in terms of marker detection, marker identification, and calibration accuracy. Our proposed system demonstrates high registration accuracy in real-life scenarios, highlighting its potential and suitability in clinical settings.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aharon, S., et al.: Shani-Perl, Roikoren755, Eran-Deci, Daniel-Deci, Shanibenbaruch: Deci-AI/super-gradients: 3.0.8 (2023).https://doi.org/10.5281/ZENODO.7789328
Community, B.O.: Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam (2018). http://www.blender.org
Dijkstra, E.W.: A note on two problems in connexion with graphs. In: Edsger Wybe Dijkstra: His Life, Work, and Legacy, pp. 287–290 (2022)
Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Medina-Carnicer, R.: Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recogn. 51, 481–491 (2016). Publisher: Elsevier
Hu, D., DeTone, D., Malisiewicz, T.: Deep ChArUco: Dark ChArUco marker pose estimation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8428–8436. IEEE, Long Beach, CA, USA (2019). https://doi.org/10.1109/CVPR.2019.00863
Li, A., Wang, L., Wu, D.: Simultaneous robot-world and hand-eye calibration using dual-quaternions and Kronecker product. Int. J. Phys. Sci. 5(10), 1530–1536 (2010)
Lin, M.A., Siu, A.F., Bae, J.H., Cutkosky, M.R., Daniel, B.L.: HoloNeedle: augmented reality guidance system for needle placement investigating the advantages of three-dimensional needle shape reconstruction. IEEE Robot. Autom. Lett. 3(4), 4156–4162 (2018). https://doi.org/10.1109/LRA.2018.2863381
Muñoz-Salinas, R., Marín-Jimenez, M.J., Yeguas-Bolivar, E., Medina-Carnicer, R.: Mapping and localization from planar markers. Pattern Recogn. 73, 158–171 (2018). https://doi.org/10.1016/j.patcog.2017.08.010
Nicolau, S., et al.: An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med. Image Anal. 13(3), 494–506 (2009). https://doi.org/10.1016/j.media.2009.02.003
Park, B.J., Hunt, S.J., Nadolski, G.J., Gade, T.P.: Augmented reality improves procedural efficiency and reduces radiation dose for CT-guided lesion targeting: a phantom study using HoloLens 2. Sci. Rep. 10(1), 18, 620 (2020). https://doi.org/10.1038/s41598-020-75676-4
Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152. IEEE Computing Society, Quebec City, Que., Canada (2001). https://doi.org/10.1109/IM.2001.924423
Zhang, Z., Hu, Y., Yu, G., Dai, J.: DeepTag: a general framework for fiducial marker design and detection. IEEE Trans. Pattern Anal. Mach. Intell., 1–1 (2022).https://doi.org/10.1109/TPAMI.2022.3174603
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Wei, Y., Zhou, S.Z. (2024). Towards Seamless Surgical Guidance: A Robust Marker-Based Multi-camera AR Navigation System with Advanced Calibration and Detection Techniques. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_23
Download citation
DOI: https://doi.org/10.1007/978-981-97-1335-6_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-1334-9
Online ISBN: 978-981-97-1335-6
eBook Packages: Computer ScienceComputer Science (R0)