research-article

The Utilization of AR/VR in Robotic Surgery: A Study

Authors:

Sonali VyasAuthors Info & Claims

ICIMMI '22: Proceedings of the 4th International Conference on Information Management & Machine Intelligence

Article No.: 61, Pages 1 - 8

https://doi.org/10.1145/3590837.3590898

Published: 30 May 2023 Publication History

Abstract

‘The surgeon, from a console in St Joseph's Hospital in Hamilton, Canada, who operates from 400km away was a headline that shook the whole world and created a mark in robot-assisted surgeries in the medical world. Doctors are thaumaturgical; saving lives and performing complex surgeries with high stakes. But doctors are also human, so they can even make errors during those procedures which might either generate a minor complexity or result in a fatal consequence. Whereas robots, unlike humans, do not make errors. They are a perfect accomplice during tasks requiring high precision, like in difficult surgeries. But what if we add augmented reality or virtual reality to the scene? How will they affect the traditional surgical methods? Will they benefit the scenario or make it worse? In this review, we will discuss AR/VR Assisted Robotic Surgery, its effects, the benefits of AR/VR in the medical industry, and how they are completely revolutionizing the surgical process in healthcare and review.

References

[1]

Fuchs KH. Minimally invasive surgery. Endoscopy. 2002;34:154–159. [ ] [Google Scholar]

[2]

Qian, Long & Wu, Jie Ying & Maio, Simon & Navab, Nassir & Kazanzides, Peter. (2019). A Review of Augmented Reality in Robotic-Assisted Surgery. IEEE Transactions on Medical Robotics and Bionics. PP. 1-1. 10.1109/TMRB.2019.2957061.

[3]

WEE SIM, K., BAKER, B., AMIN, K., CHAN, A., PATEL, K., WONG, J. Augmented and virtual reality in surgery—the digital surgical environment: applications, limitations and legal pitfalls. Annals of Translational Medicine, North America, 4, dec. 2016.

[4]

Pessaux, P.; Diana, M.; Soler, L.; Piardi, T.; Mutter, D.; Marescaux, J. Towards cybernetic surgery: Robotic and augmented reality-assisted liver segmentectomy. Langenbeck's Arch. Surg. 2015, 400, 381–385. [Google Scholar] [CrossRef] [ ]

[5]

Al-Qattan MM, Al-Turaiki TM. Flexor tendon repair in zone 2 using a six-strand 'figure of eight' suture. J Hand Surg Eur Vol 2009;34:322-8. [Crossref] [ ]

[6]

Al-Hadithy N, Ghosh S. Smartphones and the plastic surgeon. J Plast Reconstr Aesthet Surg 2013;66:e155-61. [Crossref] [ ]

[7]

Amin K. Smartphone applications for the plastic surgery trainee. J Plast Reconstr Aesthet Surg 2011;64:1255-7. [Crossref] [ ]

[8]

J. W. Yoon, "Augmented reality for the surgeon: Systematic review", Int. J. Med. Robot. Comput. Assist. Surg., vol. 14, no. 4, 2018.

[9]

G. Guthart and J. K. Salisbury, " The intuitive ™ telesurgery system: Overview and application ", Proc. ICRA, pp. 618-621, May 2000.

[10]

A. R. Lanfranco, A. E. Castellanos, J. P. Desai and W. C. Meyers, "Robotic surgery: A current perspective", Ann. Surg., vol. 239, no. 1, pp. 14-21, 2004.

[11]

G. Turchetti, I. Palla, F. Pierotti and A. Cuschieri, "Economic evaluation of da Vinci-assisted robotic surgery: A systematic review", Surg. Endoscopy, vol. 26, no. 3, pp. 598-606, 2012.

[12]

A. M. Okamura, "Haptic feedback in robot-assisted minimally invasive surgery", Current Opin. Urol., vol. 19, no. 1, pp. 102-107, 2009.

[13]

H. R. H. Patel, A. Linares and J. V. Joseph, "Robotic and laparoscopic surgery: Cost and training", Surg. Oncol., vol. 18, no. 3, pp. 242-246, 2009.

[14]

K. Kim, M. Billinghurst, G. Bruder, H. B.-L. Duh and G. F. Welch, "Revisiting trends in augmented reality research: A review of the 2nd decade of ISMAR (2008–2017)", IEEE Trans. Vis. Comput. Graphics, vol. 24, no. 11, pp. 2947-2962, Nov. 2018.

[15]

B. Kress and T. Starner, "A review of head-mounted displays (HMD) technologies and applications for consumer electronics", Proc. SPIE, vol. 8720, 2013.

[16]

O. Bimber and R. Raskar, Spatial Augmented Reality: Merging Real and Virtual Worlds., Wellesley, MA, USA:AK Peters, 2005.

[17]

J. W. Yoon, "Augmented reality for the surgeon: Systematic review", Int. J. Med. Robot. Comput. Assist. Surg., vol. 14, no. 4, 2018.

[18]

L. Qian, J. Y. Wu, S. P. DiMaio, N. Navab and P. Kazanzides, "A Review of Augmented Reality in Robotic-Assisted Surgery," in IEEE Transactions on Medical Robotics and Bionics, vol. 2, no. 1, pp. 1-16, Feb. 2020.

[19]

Pessaux, P.; Diana, M.; Soler, L.; Piardi, T.; Mutter, D.; Marescaux, J. Towards cybernetic surgery: Robotic and augmented reality-assisted liver segmentectomy. Langenbeck's Arch. Surg. 2015, 400, 381–385. [Google Scholar] [CrossRef] [ ]

[20]

Satava RM. Surgical robotics: the early chronicles: a personal historical perspective. Surg Laparosc Endosc Percutan Tech. 2002;12:6–16. [ ] [Google Scholar]

[21]

Felger JE, Nifong L. The evolution of and early experience with robot assisted mitral valve surgery. Surg Laparosc Endosc Percutan Tech. 2002;12:58–63. [ ] [Google Scholar]

[22]

Marescaux J, Leroy J, Rubino F, Transcontinental robot-assisted remote telesurgery: feasibility and potential applications. Ann Surg. 2002;235:487–492. [PMC free article] [ ] [Google Scholar]

[23]

Cheah WK, Lee B, Lenzi JE, Telesurgical laparoscopic cholecystectomy between two countries. Surg Endosc. 2000;14:1085. [ ] [Google Scholar]

[24]

Jones SB, Jones DB. Surgical aspects and future developments in laparoscopy. Anesthiol Clin North Am. 2001;19:107–124. [ ] [Google Scholar]

[25]

Kim VB, Chapman WH, Albrecht RJ, Early experience with telemanipulative robot-assisted laparoscopic cholecystectomy using Da Vinci. Surg Laparosc Endosc Percutan Tech. 2002;12:34–40. [ ] [Google Scholar]

[26]

Prasad SM, Ducko CT, Stephenson ER, Prospective clinical trial of robotically assisted endoscopic coronary grafting with 1 year follow-up. Ann Surg. 2001;233:725–732. [PMC free article] [ ] [Google Scholar]

[27]

Kwoh YS, Hou J, Jonckheere EA, . A robot with improved absolute positioning accuracy for CT guided stereotactic brain surgery. IEEE Trans Biomed Eng. 1988;35:153–161. [ ] [Google Scholar]

[28]

Fuchs KH. Minimally invasive surgery. Endoscopy. 2002;34:154–159. [ ] [Google Scholar]

[29]

Davies B. A review of robotics in surgery. Proc Inst Mech Eng. 2000;214:129–140. [ ] [Google Scholar]

[30]

R. H. Taylor, A. Menciassi, G. Fichtinger, P. Fiorini and P. Dario, "Medical robotics and computer-integrated surgery" in Handbook of Robotics, Cham, Switzerland:Springer, pp. 1657-1684, 2016.

[31]

Felger JE, Nifong L. The evolution of and early experience with robot assisted mitral valve surgery. Surg Laparosc Endosc Percutan Tech. 2002;12:58–63. [ ] [Google Scholar]

[32]

Miyake RK, Zeman HD, Duarte FH, Vein imaging: a new method of near-infrared imaging, where a processed image is projected onto the skin for the enhancement of vein treatment. Dermatol Surg 2006;32:1031-8. [ ]

[33]

Reidsma D, Katayose H, Nijholt A. editors. Advances in computer entertainment: 10th international conference, ACE 2013 Boekelo, The Netherlands, November 12-15, 2013, proceedings. Berlin: Springer, 2013.

[34]

Alberti O, Dorward NL, Kitchen ND, Neuronavigation–impact on operating time. Stereotact Funct Neurosurg 1997;68:44-8. [Crossref] [ ]

[35]

Klopfer E, Sheldon J. Augmenting your reality: student authoring of science-based augmented reality games. New Dir Youth Dev 2010;2010:85-94.

[36]

Liu, W.P.; Richmon, J.D.; Sorger, J.M.; Azizian, M.; Taylor, R.H. Augmented reality and cone beam CT guidance for transoral robotic surgery. J. Robot. Surg. 2015, 9, 223–233. [Google Scholar] [CrossRef] [ ][Green Version]

[37]

Navab, N.; Hennersperger, C.; Frisch, B.; Fürst, B. Personalized, relevance-based multimodal robotic imaging and augmented reality for computer assisted interventions. Med Image Anal. 2016, 33, 64–71. [Google Scholar] [CrossRef] [ ]

[38]

Coleman J, Nduka CC, Darzi A. Virtual reality and laparoscopic surgery. Br J Surg 1994;81:1709-11. [Crossref] [ ]

[39]

Angeles JG, Heminger H, Mass DP. Comparative biomechanical performances of 4-strand core suture repairs for zone II flexor tendon lacerations. J Hand Surg Am 2002;27:508-17. [Crossref] [ ]

[40]

Medical Realities. Press article library. Available online: http://www.medicalrealities.com/press-page/

[41]

J. Steuer, Defining virtual reality: dimensions determining telepresence, J Commun, 42 (4) (1992), pp. 73-93, 10.1111/j.1460-2466.1992.tb00812.x

[42]

P. Milgram, F. Kishino, A taxonomy of mixed reality visual displays, IEICE Trans Inf Syst., E77-D (12) (1994), pp. 1321-1329

[43]

Hu H, Shao Z, Ye L, Jin H. “Application of mixed reality technology in surgery,” 2019. /paper/Application-of-mixed-reality-technology-in-surgery-Hu-Shao/33f3959fa33697f55f78c65bf3603d5cc77c0f1c (accessed Jan. 07, 2021).

[44]

D. Umebayashi, Y. Yamamoto, Y. Nakajima, N. Fukaya, M. Hara, Augmented reality visualization-guided microscopic spine surgery: transvertebral anterior cervical foraminotomy and posterior foraminotomy, J Am Acad Orthop Surg Glob Res Rev, 2 (4) (2018), 10.5435/JAAOSGlobal-D-17-00008e008

[45]

Gumaa, A Youssef Rehan. Is virtual reality effective in orthopedic rehabilitation? A systematic review and meta-analysis Phys Ther, 99 (10) (2019), pp. 1304-1325, 10.1093/ptj/pzz093

[46]

A. Martin-Gomez, C Hill, Y Lin, J Fotouhi, S Han-Oh, KH Wang, Towards exploring the benefits of augmented reality for patient support during radiation oncology interventions Comput Methods Biomech Biomed Eng Imaging Vis, 0 (0) (2020), pp. 1-8, 10.1080/21681163.2020.1835547

[47]

P. Vávra, J Roman, P Zonča, P Ihnát, M Němec, J Kumar, Recent development of augmented reality in surgery: a review J Healthc Eng (2017) http://www.hindawi.com/journals/jhe/2017/4574172/ (accessed Jan. 07, 2021)

[48]

B.A. Ponce, J.K. Jennings, T.B. Clay, M.B. May, C. Huisingh, E.D. Sheppard Telementoring: use of augmented reality in orthopaedic education: AAOS exhibit selection J Bone Joint Surg Am, 96 (10) (2014), p. e84, 10.2106/JBJS.M.00928

[49]

D. Chytas, M.-A. Malahias, V.S. Nikolaou Augmented reality in orthopedics: current state and future directions Front Surg., 6 (2019), 10.3389/fsurg.2019.00038

[50]

P. Fallavollita, L. Wang, S. Weidert, N. Navab Augmented reality in orthopaedic interventions and education Lect Notes Comput Vis Biomech, 23 (2015), pp. 251-269, 10.1007/978-3-319-23482-3_13+

[51]

M. Blackwell, F. Morgan, A. DiGioia Augmented reality and its future in orthopaedics Clin Orthop., 354 (1998), pp. 111-122, 10.1097/00003086-199809000-00014

[52]

G.D. Stetten, V.S. Chib, Overlaying ultrasonographic images on direct vision, J Ultrasound Med Off J Am Inst Ultrasound Med, 20 (3) (2001), pp. 235-240, 10.7863/jum.2001.20.3.235

[53]

H.S. Cho, M Park, G Sanjay, I Han, H-S Kim, H Choi, Can augmented reality be helpful in pelvic bone cancer surgery? An in vitro study, Clin Orthop, 476 (9) (2018), pp. 1719-1725, 10.1007/s11999.0000000000000233

[54]

H.S. Cho, Y.K. Park, S. Gupta, C. Yoon, I. Han, H.S. Kim, Augmented reality in bone tumour resection: an experimental study, Bone Jt Res, 6 (3) (2017), pp. 137-143, 10.1302/2046-3758.63.BJR-2016-0289.R1

[55]

N. Befrui, . Fischer, B Fuerst, S-C Lee, J. Fotouhi, S Weidert„3D-augmented-reality“-Visualisierung für die navigierte Osteosynthese von Beckenfrakturen, Unfallchirurg, 121 (4) (2018), pp. 264-270, 10.1007/s00113-018-0466-y

[56]

Close, C. Bichlmeier, S.M. Heining, M. Feuerstein, N. Navab, The virtual mirror: a new interaction paradigm for augmented reality environments, IEEE Trans Med Imaging, 28 (9) (2009), pp. 1498-1510, 10.1109/TMI.2009.2018622)

[57]

D. Umebayashi, Y. Yamamoto, Y. Nakajima, N. Fukaya, M. Hara, Augmented reality visualization-guided microscopic spine surgery: transvertebral anterior cervical foraminotomy and posterior foraminotomy, J Am Acad Orthop Surg Glob Res Rev, 2 (4) (2018), 10.5435/JAAOSGlobal-D-17-00008e008

[58]

Bichlmeier, C., Heining, S.M., Feuerstein, M., & Navab, N. (2009). The Virtual Mirror: A New Interaction Paradigm for Augmented Reality Environments. IEEE Transactions on Medical Imaging, 28, 1498-1510.

Cited By

Liu CLiu YHong FGuo ZYu JLi F(2024)Mring: Contactless MR/AR Extended Input Method Based on Magnetic SignalACM Transactions on Sensor Networks10.1145/3708345Online publication date: 11-Dec-2024
https://dl.acm.org/doi/10.1145/3708345
Wang HDo THan Z(2024)Designing Indicators to Show a Robot's Physical Vision Capability2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00290(987-988)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00290
Somayaji S(2024)The Metaverse and Its Related Technologies in Telesurgery in Healthcare 5.0: A Case StudyThe Metaverse for the Healthcare Industry10.1007/978-3-031-60073-9_13(257-274)Online publication date: 2-May-2024
https://doi.org/10.1007/978-3-031-60073-9_13
Show More Cited By

Recommendations

Extended Reality (XR) Toward Building Immersive Solutions: The Key to Unlocking Industry 4.0
When developing XR applications for Industry 4.0, it is important to consider the integration of visual displays, hardware components, and multimodal interaction techniques that are compatible with the entire system. The potential use of multimodal ...
Comparative Study of HMD-based Virtual and Augmented Realities for Immersive Museums: User Acceptance, Medium, and Learning
Technologies like Head Mounted Display (HMD)-based Virtual Reality (VR) and Augmented Reality (AR) have made HMD-based immersive museums possible. To investigate the user acceptance, medium, and interaction experience for HMD-based immersive museums, an ...
Regulation of the Metaverse: A Roadmap: The risks and regulatory solutions for largescale consumer platforms.
ICVARS '22: Proceedings of the 2022 6th International Conference on Virtual and Augmented Reality Simulations

Over the last thirty years, virtual reality (VR) and augmented reality (AR) technologies have steadily advanced, enabling high-fidelity experiences at consumer prices. Over the same period, network speeds have risen dramatically. Combined, such ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

ICIMMI '22: Proceedings of the 4th International Conference on Information Management & Machine Intelligence

December 2022

749 pages

ISBN:9781450399937

DOI:10.1145/3590837

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 May 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

ICIMMI 2022

ICIMMI 2022: International Conference on Information Management & Machine Intelligence

December 23 - 24, 2022

Jaipur, India

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

4
Total Citations
View Citations
172
Total Downloads

Downloads (Last 12 months)89
Downloads (Last 6 weeks)19

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu CLiu YHong FGuo ZYu JLi F(2024)Mring: Contactless MR/AR Extended Input Method Based on Magnetic SignalACM Transactions on Sensor Networks10.1145/3708345Online publication date: 11-Dec-2024
https://dl.acm.org/doi/10.1145/3708345
Wang HDo THan Z(2024)Designing Indicators to Show a Robot's Physical Vision Capability2024 IEEE Conference on Virtual Reality and 3D User Interfaces Abstracts and Workshops (VRW)10.1109/VRW62533.2024.00290(987-988)Online publication date: 16-Mar-2024
https://doi.org/10.1109/VRW62533.2024.00290
Somayaji S(2024)The Metaverse and Its Related Technologies in Telesurgery in Healthcare 5.0: A Case StudyThe Metaverse for the Healthcare Industry10.1007/978-3-031-60073-9_13(257-274)Online publication date: 2-May-2024
https://doi.org/10.1007/978-3-031-60073-9_13
Chakravarthi BM PImandi RN P(2023)A Comprehensive Review of Leap Motion Controller-Based Hand Gesture Datasets2023 International Conference on Next Generation Electronics (NEleX)10.1109/NEleX59773.2023.10421030(1-7)Online publication date: 14-Dec-2023
https://doi.org/10.1109/NEleX59773.2023.10421030

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten