Skip to main content

Advertisement

SpringerLink
Log in

A multi-view interactive virtual-physical registration method for mixed reality based surgical navigation in pelvic and acetabular fracture fixation

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The treatment of pelvic and acetabular fractures remains technically demanding, and traditional surgical navigation systems suffer from the hand–eye mis-coordination. This paper describes a multi-view interactive virtual-physical registration method to enhance the surgeon’s depth perception and a mixed reality (MR)-based surgical navigation system for pelvic and acetabular fracture fixation.

Methods

First, the pelvic structure is reconstructed by segmentation in a preoperative CT scan, and an insertion path for the percutaneous LC-II screw is computed. A custom hand-held registration cube is used for virtual-physical registration. Three strategies are proposed to improve the surgeon’s depth perception: vertices alignment, tremble compensation and multi-view averaging. During navigation, distance and angular deviation visual cues are updated to help the surgeon with the guide wire insertion. The methods have been integrated into an MR module in a surgical navigation system.

Results

Phantom experiments were conducted. Ablation experimental results demonstrated the effectiveness of each strategy in the virtual-physical registration method. The proposed method achieved the best accuracy in comparison with related works. For percutaneous guide wire placement, our system achieved a mean bony entry point error of 2.76 ± 1.31 mm, a mean bony exit point error of 4.13 ± 1.74 mm, and a mean angular deviation of 3.04 ± 1.22°.

Conclusions

The proposed method can improve the virtual-physical fusion accuracy. The developed MR-based surgical navigation system has clinical application potential. Cadaver and clinical experiments will be conducted in future.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Butterwick D, Papp S, Gofton W, Liew A, Beaulé PE (2015) Acetabular fractures in the elderly: evaluation and management. J Bone Joint Surg 97(9):758–768

    Article  PubMed  Google Scholar 

  2. Wong JML, Bewsher S, Yew J, Bucknill A, Steiger R (2015) Fluoroscopically assisted computer navigation enables accurate percutaneous screw placement for pelvic and acetabular fracture fixation. Injury 46(6):1064–1068

    Article  PubMed  Google Scholar 

  3. Karkenny AJ, Mendelis JR, Geller DS, Gomez JA (2019) The role of intraoperative navigation in orthopaedic surgery. JAAOS-J Am Acad Orthopaedic Surg 27(19):e849–e858

    Article  Google Scholar 

  4. Lungu AJ, Swinkels W, Claesen L, Tu P, Egger J, Chen X (2021) A review on the applications of virtual reality, augmented reality and mixed reality in surgical simulation: an extension to different kinds of surgery. Expert Rev Med Devices 18(1):47–62

    Article  CAS  PubMed  Google Scholar 

  5. Chen X, Xu L, Wang Y, Wang H, Wang F, Zeng X, Egger J (2015) Development of a surgical navigation system based on augmented reality using an optical see-through head-mounted display. J Biomed Inform 55:124–131

    Article  PubMed  Google Scholar 

  6. Liebmann F, Roner S, Atzigen M, Scaramuzza D, Sutter R, Snedeker J, Fürnstahl P (2019) Pedicle screw navigation using surface digitization on the Microsoft HoloLens. Int J Comp Assist Radiol Surg 14(7):1157–1165.

  7. Gsaxner C, Pepe A, Li J, Ibrahimpasic U, Wallner J, Schmalstieg D, Egger J (2021) Augmented reality for head and neck carcinoma imaging: Description and feasibility of an instant calibration, markerless approach. Comput Methods Programs Biomed 200:105854

    Article  PubMed  Google Scholar 

  8. Atzigen M, Liebmann F, Hoch A, Spirig JM, Farshad M, Snedeker J, Fürnstahl P (2022) Marker-free surgical navigation of rod bending using a stereo neural network and augmented reality in spinal fusion. Med Image Anal 77:102365

    Article  Google Scholar 

  9. Gsaxner C, Pepe A, Wallner J, Schmalstieg D, Egger J (2019) Markerless image-to-face registration for untethered augmented reality in head and neck surgery. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp 236–244.

  10. Sun Q, Mai Y, Yang R, Ji T, Jiang X, Chen X (2020) Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int J Comp Assist Radiol Surg 15(11):1907–1919.

  11. Li R, Tong Y, Yang T, Guo J, Si W, Zhang Y, Heng PA (2021) Towards quantitative and intuitive percutaneous tumor puncture via augmented virtual reality. Comput Med Imaging Graph 90:101905

    Article  PubMed  Google Scholar 

  12. Song T, Yang C, Dianat O, Azimi E (2018) Endodontic guided treatment using augmented reality on a head-mounted display system. Healthcare Technol Lett 5(5):201–207

    Article  Google Scholar 

  13. Tu P, Qin C, Guo Y, Li D, Lungu AJ, Wang H, Chen X (2022) Ultrasound image guided and mixed reality-based surgical system with real-time soft tissue deformation computing for robotic cervical pedicle screw placement. IEEE Trans Biomed Eng 68(8):2593–2603

    Article  Google Scholar 

  14. Oliveira ME, Debarba HG, Lädermann A, Chagué S, Charbonnier C (2019) A hand‐eye calibration method for augmented reality applied to computer‐assisted orthopedic surgery. Int J Med Robot Computer Assisted Surg 15(2):e1969.

  15. Fotouhi J, Mehrfard A, Song T, Johnson A, Osgood G, Unberath M, Navab N (2020) Development and pre-clinical analysis of spatiotemporal-aware augmented reality in orthopedic interventions. IEEE Trans Med Imaging 40(2):765–778

    Article  Google Scholar 

  16. Scherer J, Guy P, Lefaivre KA, Pape HC, Werner CM, Osterhoff G (2017) Guide wire insertion for percutaneous LC2 screws in acetabular and pelvic ring fixation using a transpedicular working cannula. Injury 48(10):2360–2364

    Article  PubMed  Google Scholar 

  17. Wang H, Wang F, Newman S, Lin Y, Chen X, Xu L, Wang Q (2016) Application of an innovative computerized virtual planning system in acetabular fracture surgery: a feasibility study. Injury 47(8):1698–1701

    Article  PubMed  Google Scholar 

  18. Tu P, Gao Y, Lungu AJ, Li D, Wang H, Chen X (2021) Augmented reality based navigation for distal interlocking of intramedullary nails utilizing Microsoft HoloLens 2. Comput Biol Med 133:104402

    Article  PubMed  Google Scholar 

  19. Lane CG, Warren R, Pearle AD (2008) The pivot shift. JAAOS-J Am Acad Orthopaed Surgeons 16(12):679–688

    Article  Google Scholar 

  20. Moakher M (2002) Means and averaging in the group of rotations. SIAM J Matrix Anal Appl 24(1):1–16

    Article  Google Scholar 

  21. Gao Y, Wang H, Tu P, Hu J, Wang Q, Chen X (2021) A novel dynamic electromagnetic tracking navigation system for distal locking of intramedullary nails. Comput Methods Programs Biomed 209:106326

    Article  PubMed  Google Scholar 

  22. Qin C, Cao Z, Fan S, Wu Y, Sun Y, Politis C, Chen X (2019) An oral and maxillofacial navigation system for implant placement with automatic identification of fiducial points. Int J Comp Assist Radiol Surg 14(2):281–289.

  23. Fan X, Zhu Q, Tu P, Joskowicz L, Chen X (2022) A review of advances in image-guided orthopedic surgery. Phys Med Biolo 68:02TR01.

Download references

Acknowledgments

This work was supported by grants from National Key R&D Program of China (2022YFE0197900), National Natural Science Foundation of China (81971709; M-0019; 82011530141; 82202302), the Foundation of Science and Technology Commission of Shanghai Municipality (20490740700), Shanghai Jiao Tong University Foundation on Medical and Technological Joint Science Research (YG2019ZDA06; YG2021ZD21; YG2021QN72; YG2022QN056), SJTU Global Strategic Partnership Fund (2023 SJTU-CORNELL, 2021 SJTU-HUJI), and a Joint China-Israel grant from the Ministry of Science and Technology, Israel, 2021-2023.

Author information

Authors and Affiliations

  1. Institute of Biomedical Manufacturing and Life Quality Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, Shanghai, China

    Puxun Tu & Xiaojun Chen

  2. Department of Orthopedics, Shanghai Sixth People’s Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China

    Huixiang Wang

  3. School of Computer Science and Engineering, The Hebrew University of Jerusalem, Jerusalem, Israel

    Leo Joskowicz

  4. Institute of Medical Robotics, Shanghai Jiao Tong University, Shanghai, China

    Xiaojun Chen

Authors
  1. Puxun Tu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huixiang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Leo Joskowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaojun Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Huixiang Wang or Xiaojun Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards.

Informed consent

There was no informed consent required for the work reported in this manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tu, P., Wang, H., Joskowicz, L. et al. A multi-view interactive virtual-physical registration method for mixed reality based surgical navigation in pelvic and acetabular fracture fixation. Int J CARS 18, 1715–1724 (2023). https://doi.org/10.1007/s11548-023-02884-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-02884-4

Keywords

Search

Navigation