Skip to main content
Springer Nature Link
Log in

Natural 3D Object Manipulation for Interactive Laparoscopic Augmented Reality Registration

  • Conference paper
  • First Online:
Virtual, Augmented and Mixed Reality: Design and Development (HCII 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13317))

Included in the following conference series:

  • 2492 Accesses

Abstract

Due to the growing focus on minimally invasive surgery, there is increasing interest in intraoperative software support. For example, augmented reality can be used to provide additional information. Accurate registration is required for effective support. In this work, we present a manual registration method that aims at mimicking natural manipulation of 3D objects using tracked surgical instruments. This method is compared to a point-based registration method in a simulated laparoscopic environment. Both registration methods serve as an initial alignment step prior to surface-based registration refinement. For the evaluation, we conducted a user study with 12 participants. The registration methods were compared in terms of registration accuracy, registration duration, and subjective usability feedback. No significant differences could be found with respect to the previously mentioned criteria between the manual and the point-based registration methods. Thus, the manual registration did not outperform the reference method. However, we found that our method offers qualitative advantages, which may make it more suitable for some application scenarios. Furthermore we identified possible approaches for improvement, which should be investigated in the future to strengthen possible advantages of our registration method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Arun, K.S., Huang, T.S., Blostein, S.D.: Least-squares fitting of two 3-d point sets. IEEE Trans. Pattern Anal. Mach. Intell. PAMI-9(5), 698–700 (1987). https://doi.org/10.1109/TPAMI.1987.4767965

  2. Benincasa, A.B., Clements, L.W., Herrell, S.D., Galloway, R.L.: Feasibility study for image-guided kidney surgery: assessment of required intraoperative surface for accurate physical to image space registrations. Med. Phys. 35(9), 4251–4261 (2008)

    Article  Google Scholar 

  3. Bernhardt, S., Nicolau, S.A., Soler, L., Doignon, C.: The status of augmented reality in laparoscopic surgery as of 2016. Med. Image Anal. 37, 66–90 (2017)

    Article  Google Scholar 

  4. Besl, P.J., McKay, N.D.: Method for registration of 3-d shapes. In: Sensor Fusion IV: Control Paradigms and Data Structures, vol. 1611, pp. 586–606. International Society for Optics and Photonics (1992)

    Google Scholar 

  5. Bradski, G.: The OpenCV library. Dr. Dobb’s J. Softw. Tools Prof. Program. 25(11), 120–123 (2000)

    Google Scholar 

  6. Brooke, J., et al.: SUS - a quick and dirty usability scale. Usability Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  7. Chen, Y., Li, H., Wu, D., Bi, K., Liu, C.: Surgical planning and manual image fusion based on 3d model facilitate laparoscopic partial nephrectomy for intrarenal tumors. World J. Urol. 32(6), 1493–1499 (2014)

    Article  Google Scholar 

  8. Conrad, C., Fusaglia, M., Peterhans, M., Lu, H., Weber, S., Gayet, B.: Augmented reality navigation surgery facilitates laparoscopic rescue of failed portal vein embolization. J. Am. Coll. Surg. 223(4), e31–e34 (2016)

    Article  Google Scholar 

  9. De Araújo, B.R., Casiez, G., Jorge, J.A., Hachet, M.: Mockup builder: 3d modeling on and above the surface. Comput. Graph. 37(3), 165–178 (2013)

    Article  Google Scholar 

  10. Garrido-Jurado, S., Muñoz-Salinas, R., Madrid-Cuevas, F.J., Marín-Jiménez, M.J.: Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn. 47(6), 2280–2292 (2014)

    Article  Google Scholar 

  11. Gong, R.H., Güler, Ö., Kürklüoglu, M., Lovejoy, J., Yaniv, Z.: Interactive initialization of 2d/3d rigid registration. Med. Phys. 40(12), 121911 (2013)

    Article  Google Scholar 

  12. Heller, N., et al.: The KiTS19 challenge data: 300 kidney tumor cases with clinical context, CT semantic segmentations, and surgical outcomes. arXiv preprint arXiv:1904.00445 (2019)

  13. Joeres, F., Heinrich, F., Schott, D., Hansen, C.: Towards natural 3d interaction for laparoscopic augmented reality registration. Comput. Meth. Biomech. Biomed. Eng. Imaging Visual. 9(4), 384–391 (2020)

    Article  Google Scholar 

  14. Joeres, F., Mielke, T., Hansen, C.: Laparoscopic augmented reality registration for oncological resection site repair. Int. J. Comput. Assist. Radiol. Surg. 16(9), 1577–1586 (2021). https://doi.org/10.1007/s11548-021-02336-x

  15. Mendes, D., Caputo, F.M., Giachetti, A., Ferreira, A., Jorge, J.: A survey on 3d virtual object manipulation: from the desktop to immersive virtual environments. In: Computer Graphics Forum, vol. 38, pp. 21–45 (2019)

    Google Scholar 

  16. Nicolau, S., Soler, L., Mutter, D., Marescaux, J.: Augmented reality in laparoscopic surgical oncology. Surg. Oncol. 20(3), 189–201 (2011)

    Article  Google Scholar 

  17. Pratt, P.: An effective visualisation and registration system for image-guided robotic partial nephrectomy. J. Robot. Surg. 6(1), 23–31 (2012)

    Article  Google Scholar 

  18. Schneider, C.: Comparison of manual and semi-automatic registration in augmented reality image-guided liver surgery: a clinical feasibility study. Surg. Endosc. 34(10), 4702–4711 (2020). https://doi.org/10.1007/s00464-020-07807-x

    Article  Google Scholar 

  19. Scott, D.J.: Laparoscopic training on bench models: better and more cost effective than operating room experience? J. Am. Coll. Surg. 191(3), 272–283 (2000)

    Article  Google Scholar 

  20. Song, P., Goh, W.B., Hutama, W., Fu, C.W., Liu, X.: A handle bar metaphor for virtual object manipulation with mid-air interaction. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 1297–1306 (2012)

    Google Scholar 

  21. Thompson, S., et al.: Accuracy validation of an image guided laparoscopy system for liver resection. In: Medical imaging 2015: Image-Guided Procedures, Robotic Interventions, and Modeling, vol. 9415, p. 941509. International Society for Optics and Photonics (2015)

    Google Scholar 

  22. Zhang, Z.: A flexible new technique for camera calibration. IEEE Trans. Pattern Anal. Mach. Intell. 22(11), 1330–1334 (2000)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Research Campus STIMULATE, Otto-von-Guericke-University, Magdeburg, Germany

    Tonia Mielke, Fabian Joeres & Christian Hansen

Authors
  1. Tonia Mielke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabian Joeres
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Christian Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christian Hansen .

Editor information

Editors and Affiliations

  1. U.S. Army Research Laboratory, Aberdeen Proving Ground, MD, USA

    Jessie Y. C. Chen

  2. U.S. Army Combat Capabilities Development Command Soldier Center, Orlando, FL, USA

    Gino Fragomeni

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Mielke, T., Joeres, F., Hansen, C. (2022). Natural 3D Object Manipulation for Interactive Laparoscopic Augmented Reality Registration. In: Chen, J.Y.C., Fragomeni, G. (eds) Virtual, Augmented and Mixed Reality: Design and Development. HCII 2022. Lecture Notes in Computer Science, vol 13317. Springer, Cham. https://doi.org/10.1007/978-3-031-05939-1_21

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-05939-1_21

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-05938-4

  • Online ISBN: 978-3-031-05939-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics