Skip to main content

Towards Seamless Surgical Guidance: A Robust Marker-Based Multi-camera AR Navigation System with Advanced Calibration and Detection Techniques

  • Conference paper
  • First Online:
Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023) (MICAD 2023)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 1166))

  • 413 Accesses

Abstract

In recent years, many marker-based augmented reality (AR) surgical navigation systems have been developed to assist with surgeries. However, the current marker-based registration methods encounter challenges related to complex marker placement and adjustments, limiting their widespread adoption in clinical settings, especially in diverse surgical scenarios. This study presents a novel marker-based multi-camera AR surgical navigation system that encompasses a robust marker detection framework and a high-precision registration system. Our marker-based registration system addresses the challenges of marker placement by utilizing an extended perception range and a high-accuracy calibration solution. Moreover, we introduce an innovative deep learning-based framework designed to provide accurate and robust marker detection even under challenging lighting and motion conditions. Validation experiments underscore the superiority of our system over standard methods, exhibiting enhanced performance in terms of marker detection, marker identification, and calibration accuracy. Our proposed system demonstrates high registration accuracy in real-life scenarios, highlighting its potential and suitability in clinical settings.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Aharon, S., et al.: Shani-Perl, Roikoren755, Eran-Deci, Daniel-Deci, Shanibenbaruch: Deci-AI/super-gradients: 3.0.8 (2023).https://doi.org/10.5281/ZENODO.7789328

  2. Community, B.O.: Blender - a 3D modelling and rendering package. Blender Foundation, Stichting Blender Foundation, Amsterdam (2018). http://www.blender.org

  3. Dijkstra, E.W.: A note on two problems in connexion with graphs. In: Edsger Wybe Dijkstra: His Life, Work, and Legacy, pp. 287–290 (2022)

    Google Scholar 

  4. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Medina-Carnicer, R.: Generation of fiducial marker dictionaries using mixed integer linear programming. Pattern Recogn. 51, 481–491 (2016). Publisher: Elsevier

    Google Scholar 

  5. Hu, D., DeTone, D., Malisiewicz, T.: Deep ChArUco: Dark ChArUco marker pose estimation. In: 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8428–8436. IEEE, Long Beach, CA, USA (2019). https://doi.org/10.1109/CVPR.2019.00863

  6. Li, A., Wang, L., Wu, D.: Simultaneous robot-world and hand-eye calibration using dual-quaternions and Kronecker product. Int. J. Phys. Sci. 5(10), 1530–1536 (2010)

    Google Scholar 

  7. Lin, M.A., Siu, A.F., Bae, J.H., Cutkosky, M.R., Daniel, B.L.: HoloNeedle: augmented reality guidance system for needle placement investigating the advantages of three-dimensional needle shape reconstruction. IEEE Robot. Autom. Lett. 3(4), 4156–4162 (2018). https://doi.org/10.1109/LRA.2018.2863381

    Article  Google Scholar 

  8. Muñoz-Salinas, R., Marín-Jimenez, M.J., Yeguas-Bolivar, E., Medina-Carnicer, R.: Mapping and localization from planar markers. Pattern Recogn. 73, 158–171 (2018). https://doi.org/10.1016/j.patcog.2017.08.010

    Article  Google Scholar 

  9. Nicolau, S., et al.: An augmented reality system for liver thermal ablation: design and evaluation on clinical cases. Med. Image Anal. 13(3), 494–506 (2009). https://doi.org/10.1016/j.media.2009.02.003

    Article  Google Scholar 

  10. Park, B.J., Hunt, S.J., Nadolski, G.J., Gade, T.P.: Augmented reality improves procedural efficiency and reduces radiation dose for CT-guided lesion targeting: a phantom study using HoloLens 2. Sci. Rep. 10(1), 18, 620 (2020). https://doi.org/10.1038/s41598-020-75676-4

  11. Rusinkiewicz, S., Levoy, M.: Efficient variants of the ICP algorithm. In: Proceedings Third International Conference on 3-D Digital Imaging and Modeling, pp. 145–152. IEEE Computing Society, Quebec City, Que., Canada (2001). https://doi.org/10.1109/IM.2001.924423

  12. Zhang, Z., Hu, Y., Yu, G., Dai, J.: DeepTag: a general framework for fiducial marker design and detection. IEEE Trans. Pattern Anal. Mach. Intell., 1–1 (2022).https://doi.org/10.1109/TPAMI.2022.3174603

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Yizhi Wei .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wei, Y., Zhou, S.Z. (2024). Towards Seamless Surgical Guidance: A Robust Marker-Based Multi-camera AR Navigation System with Advanced Calibration and Detection Techniques. In: Su, R., Zhang, YD., Frangi, A.F. (eds) Proceedings of 2023 International Conference on Medical Imaging and Computer-Aided Diagnosis (MICAD 2023). MICAD 2023. Lecture Notes in Electrical Engineering, vol 1166. Springer, Singapore. https://doi.org/10.1007/978-981-97-1335-6_23

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-1335-6_23

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-1334-9

  • Online ISBN: 978-981-97-1335-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics