Skip to main content

Advertisement

SpringerLink
Log in

Research on registration and navigation technology of augmented reality for ex-vivo hepatectomy

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

The application of augmented reality technology to the partial hepatectomy procedure has high practical significance, but the existing augmented reality navigation system has major drawbacks in the display and registration methods, which result in low precision. The augmented reality surgical navigation system proposed in this study has been improved in the above two aspects, which can significantly improve the surgical accuracy.

Methods

The use of optical see-through head-mounted displays for imaging displays can prevent doctors from reconstructing the patient's two-dimensional image information in their minds and reduce the psychological burden of doctors. In the registration process, the biomechanical properties of the liver are introduced, and a non-rigid registration method based on biomechanics is proposed and realized by a meshless algorithm. In addition, this study uses the moving grid algorithm to carry out clinical experiments on ex-vivo pig liver for experimental verification.

Results

The mark-based interactive registration error is 4.21 ± 1.6 mm, and the registration error is reduced after taking the biomechanical properties of the liver into account, which is − 0.153 ± 0.398 mm. The cutting error of the liver model is 0.159 ± 0.292 mm. In addition, with the aid of the navigation system proposed in this paper, the experiment of ex-vivo pig liver cutting was completed with an error of − 1.164 ± 0.576 mm.

Conclusions

As a proof-of-concept study, the augmented reality navigation system proposed in this study improves the traditional image-guided surgery in terms of display and registration methods, and the feasibility of the system is verified by ex-vivo pig liver experiments. Therefore, the navigation system has a certain guiding significance for clinical surgery.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig.8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig.14
Fig.15

Similar content being viewed by others

References

  1. Global, regional, and national age–sex specific all-cause and cause-specific mortality for 240 causes of death, 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013. The Lancet 385(9963): 117–171

  2. Fong ZV, Tanabe KK (2014) The clinical management of hepatocellular carcinoma in the United States, Europe, and Asia: a comprehensive and evidence-based comparison and review. Cancer 120(18):2824–2838

    Article  Google Scholar 

  3. Vermaas M, Ferenschild FTJ, Verhoef C, Nuyttens JJME, Marinelli AWKS, Wiggers T, Kirkels WJ, Eggermont AMM, De Wilt JHW (2007) Total pelvic exenteration for primary locally advanced and locally recurrent rectal cancer. Eur J Surg Oncol (EJSO) 33(4):452–458

    Article  CAS  Google Scholar 

  4. Winne C, Khan M, Stopp F, Jank E, Keeve E (2011) Overlay visualization in endoscopic ENT surgery. Int J Comput Assist Radiol Surg 6(3):401–406

    Article  Google Scholar 

  5. Mochizuki Y, Hosaka A, Kamiuchi H, Nie JX, Masamune K, Hoshina K, Miyata T, Watanabe T (2016) New simple image overlay system using a tablet PC for pinpoint identification of the appropriate site for anastomosis in peripheral arterial reconstruction. Surg Today 46(12):1387–1393

    Article  Google Scholar 

  6. Katić D, Spengler P, Bodenstedt S, Castrillon-Oberndorfer G, Seeberger R, Hoffmann J, Dillmann R, Speidel S (2015) A system for context-aware intraoperative augmented reality in dental implant surgery. Int J Comput Assist Radiol Surg 10(1):101–108

    Article  Google Scholar 

  7. Cabrilo I, Schaller K, Bijlenga P (2015) Augmented reality-assisted bypass surgery: embracing minimal invasiveness. World Neurosurg 83(4):596–602

    Article  Google Scholar 

  8. Ntourakis D, Memeo R, Soler L, Marescaux J, Mutter D, Pessaux P (2016) Augmented reality guidance for the resection of missing colorectal liver metastases: an initial experience. World J Surg 40(2):419–426

    Article  Google Scholar 

  9. Pessaux P, Diana M, Soler L, Piardi T, Mutter D, Marescaux J (2015) Towards cybernetic surgery: robotic and augmented reality-assisted liver segmentectomy. Langenbecks Arch Surg 400(3):381–385

    Article  Google Scholar 

  10. Soler L, Nicolau S, Pessaux P, Mutter D, Marescaux J (2017) Augmented reality in minimally invasive digestive surgery. In: Lima M (ed) Pediatric digestive surgery. Springer, Cham, pp 421–432

    Chapter  Google Scholar 

  11. Heizmann O, Zidowitz S, Bourquain H, Potthast S, Peitgen H-O, Oertli D, Kettelhack C (2010) Assessment of intraoperative liver deformation during hepatic resection: prospective clinical study. World J Surg 34(8):1887–1893

    Article  Google Scholar 

  12. Bao P, Sinha T, Chen CC, Warmath J, Galloway R, Herline A (2007) A prototype ultrasound-guided laparoscopic radiofrequency ablation system. Surg Endosc 21:74–79

    Article  CAS  Google Scholar 

  13. Beller S, Hünerbein M, Fau Lange T, Lange T, Fau Eulenstein S, Eulenstein S, Fau Gebauer B, Gebauer B, Schlag F, Schlag PM (2007) Image-guided surgery of liver metastases by three-dimensional ultrasound-based optoelectronic navigation. Br J Surg 94:866–875

    Article  CAS  Google Scholar 

  14. Lange T, Papenberg N, Heldmann S, Modersitzki J, Fischer B, Lamecker H, Schlag PM (2009) 3D ultrasound-CT registration of the liver using combined landmark-intensity information. Int J Comput Assist Radiol Surg 4(1):79–88

    Article  Google Scholar 

  15. Martin RCG, Husheck S, Scoggins CR, Mcmasters KM (2006) Intraoperative magnetic resonance imaging for ablation of hepatic tumors. Surg Endosc Other Int Tech 20(10):1536–1542

    Article  CAS  Google Scholar 

  16. Haouchine N, Cotin S, Peterlik I, Dequidt J, Lopez MS, Kerrien E, Berger M (2015) Impact of soft tissue heterogeneity on augmented reality for liver surgery. IEEE Trans Visual Comput Graphics 21(5):584–597

    Article  Google Scholar 

  17. Sun Q, Mai Y, Yang R, Tong J, Xiaojun C (2020) Fast and accurate online calibration of optical see-through head-mounted display for AR-based surgical navigation using Microsoft HoloLens. Int J CARS 15:1907–1919

    Article  Google Scholar 

  18. Grubert J, Itoh Y, Moser K, Swan JE (2018) A survey of calibration methods for optical see-through head-mounted displays. IEEE Trans Visual Comput Graph 24(9):2649–2662

    Article  Google Scholar 

  19. Wagner D, Schmalstieg D (2007) ARToolKitPlus for pose tracking on mobile devices. In: Proceedings of 12th computer vision winter workshop CVWW07

  20. Gingold RA, Monaghan JJ (1977) Smoothed particle hydrodynamics: theory and application to non-spherical stars. Mon Not R Astron Soc 181:375–389

    Article  Google Scholar 

  21. Kawashima Y, Sakai Y (2007) Large deformation analysis of hyperelastic materials using SPH method. e-J Soft Mater 3:21–28

    Article  Google Scholar 

  22. Vakilha M, Manzari MT (2008) Modelling of power-law fluid flow through porous media using smoothed particle hydrodynamics. Transp Porous Media 74(3):331–346

    Article  Google Scholar 

  23. Rafiee A, Manzari MT, Hosseini M (2007) An incompressible SPH method for simulation of unsteady viscoelastic free-surface flows. Int J Non-Linear Mech 42(10):1210–1223

    Article  Google Scholar 

  24. Goktekin TG, Bargteil AW, Brien JF (2004) A method for animating viscoelastic fluids. ACM Trans. Graph. (SIGGRAPH) 23(3):463–468

    Article  Google Scholar 

  25. Steele K, Cline D, Egbert PK, Dinerstein J (2004) Modeling and rendering viscous liquids. Comput Anim Virtual Worlds 15(3–4):183–192

    Article  Google Scholar 

  26. Batty C, Uribe A, Audoly B, Grinspun E (2012) Discrete viscous sheets. 31(4%J ACM Trans Graph): Article 113.

  27. Shi W, Liu PX, Zheng M (2019) A new volumetric geometric model for cutting procedures in surgical simulation. Comput Methods Programs Biomed 178:77–84

    Article  Google Scholar 

  28. Wu J, Westermann R, Dick C (2015) A survey of physically based simulation of cuts in deformable bodies. Comput Graph Forum 34:161–187

    Article  Google Scholar 

  29. Rucker DC, Wu Y, Clements LW, Ondrake JE, Pheiffer TS, Simpson AL, Jarnagin WR, Miga MI (2014) A Mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imag 33(1):147–158

    Article  Google Scholar 

  30. Zhang F, Zhang S, Zhong K et al (2020) Design of navigation system for liver surgery guided by augmented reality. IEEE Access 8:126687–126699

    Article  Google Scholar 

  31. Azimi E, Qian L, Navab N et al (2017) Alignment of the virtual scene to the tracking space of a mixed reality head-mounted display. arXiv preprint arXiv:1703.05834

  32. Rucker DC, Wu Y, Clements LW et al (2014) A mechanics-based nonrigid registration method for liver surgery using sparse intraoperative data. IEEE Trans Med Imag 33(1):147–158

    Article  Google Scholar 

Download references

Funding

The authors extend their gratitude to Soochow University Robotics and Microsystems Control Center. The authors would like to thank the National Key Research and Development Project (2018YFB1307700) for its support.

Author information

Authors and Affiliations

  1. School of Mechanical and Electrical Engineering, Soochow University, Suzhou, 215006, China

    Fengfeng Zhang & Lining Sun

  2. Collaborative Innovation Center of Suzhou Nano Science and Technology, Soochow University, Suzhou, 215123, China

    Fengfeng Zhang & Lining Sun

  3. College of Mechanical and Engineering, Harbin Engineering University, Harbin, 150001, China

    Shi Zhang & Long Sun

  4. The First Affiliated Hospital of Soochow University, Suzhou, 215006, China

    Wei Zhan

Authors
  1. Fengfeng Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shi Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Long Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lining Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fengfeng Zhang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, F., Zhang, S., Sun, L. et al. Research on registration and navigation technology of augmented reality for ex-vivo hepatectomy. Int J CARS 17, 147–155 (2022). https://doi.org/10.1007/s11548-021-02531-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-021-02531-w

Keywords

Search

Navigation