Skip to main content
SpringerLink
Log in

Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Surgical skill assessment has primarily been performed using checklists or rating scales, which are prone to bias and subjectivity. To tackle this shortcoming, assessment of surgical tool motion can be implemented to objectively classify skill levels. Due to the challenges involved in motion tracking of surgical tooltips in minimally invasive surgeries, formerly used assessment approaches may not be feasible for real-world skill assessment. We proposed an assessment approach based on the virtual marker on surgical tooltips to derive the tooltip’s 3D position and introduced a novel metric for surgical skill assessment.

Methods

We obtained the 3D tooltip position based on markers placed on the tool handle. Then, we derived tooltip motion metrics to identify the metrics differentiating the skill levels for objective surgical skill assessment. We proposed a new tooltip motion metric, i.e., motion inconsistency, that can assess the skill level, and also can evaluate the stage of skill learning. In this study, peg transfer, dual transfer, and rubber band translocation tasks were included, and nine novices, five surgical residents and five attending general surgeons participated.

Results

Our analyses showed that tooltip path length (p \(\le\) 0.007) and path length along the instrument axis (p \(\le\) 0.014) differed across the three skill levels in all the tasks and decreased by skill level. Tooltip motion inconsistency showed significant differences among the three skill levels in the dual transfer (p \(=\) 0.025) and the rubber band translocation tasks (p \(=\) 0.021). Lastly, bimanual dexterity differed across the three skill levels in all the tasks (p \(\le\) 0.012) and increased by skill level.

Conclusion

Depth perception ability (indicated by shorter tooltip path lengths along the instrument axis), bimanual dexterity, tooltip motion consistency, and economical tooltip movements (shorter tooltip path lengths) are related to surgical skill. Our findings can contribute to objective surgical skill assessment, reducing subjectivity, bias, and associated costs.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Goldenberg MG (2020) Evidence that surgical performance predicts clinical outcomes. World J Urol 38:1595–1597. https://doi.org/10.1007/s00345-019-02857-w

    Article  PubMed  Google Scholar 

  2. Siddaiah-Subramanya M, To H, Haigh C (2021) The psychosocial impact of surgical complications on the operating surgeon: a scoping review. Ann. Med. Surg 67:102530. https://doi.org/10.1016/j.amsu.2021.102530

    Article  Google Scholar 

  3. Zhan C, Miller MR (2003) Excess length of stay, charges, and mortality attributable to medical injuries during hospitalization. JAMA 290:1868–1874. https://doi.org/10.1001/jama.290.14.1868

    Article  CAS  PubMed  Google Scholar 

  4. Lazzarino AI, Nagpal K, Bottle A, Faiz O, Moorthy K, Aylin P (2010) Open versus minimally invasive esophagectomy: trends of utilization and associated outcomes in England. Ann Surg 252:292–298. https://doi.org/10.1097/SLA.0b013e3181dd4e8c

    Article  PubMed  Google Scholar 

  5. Martinec DV, Gatta P, Zheng B, Denk PM, Swanström LL (2009) The trade-off between flexibility and maneuverability: task performance with articulating laparoscopic instruments. Surg Endosc 23:2697–2701. https://doi.org/10.1007/s00464-009-0462-y

    Article  PubMed  Google Scholar 

  6. Gallagher AG, Smith CD, Bowers SP, Seymour NE, Pearson A, McNatt S, Hananel D, Satava RM (2003) Psychomotor skills assessment in practicing surgeons experienced in performing advanced laparoscopic procedures. J Am Coll Surg 197:479–488. https://doi.org/10.1016/S1072-7515(03)00535-0

    Article  PubMed  Google Scholar 

  7. Pinzon D, Byrns S, Zheng B (2016) Prevailing trends in haptic feedback simulation for minimally invasive surgery. Surg Innov 23:415–421. https://doi.org/10.1177/1553350616628680

    Article  PubMed  Google Scholar 

  8. Vitish-Sharma P, Knowles J, Patel B (2011) Acquisition of fundamental laparoscopic skills: Is a box really as good as a virtual reality trainer? Int J Surg 9:659–661. https://doi.org/10.1016/j.ijsu.2011.08.009

    Article  CAS  PubMed  Google Scholar 

  9. Martin JA, Regehr G, Reznick R, Macrae H, Murnaghan J, Hutchison C, Brown M (1997) Objective structured assessment of technical skill (OSATS) for surgical residents. Br J Surg 84:273–278. https://doi.org/10.1002/bjs.1800840237

    Article  CAS  PubMed  Google Scholar 

  10. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113. https://doi.org/10.1016/j.amjsurg.2005.04.004

    Article  PubMed  Google Scholar 

  11. Goh AC, Goldfarb DW, Sander JC, Miles BJ, Dunkin BJ (2012) Global evaluative assessment of robotic skills: validation of a clinical assessment tool to measure robotic surgical skills. J Urol 187:247–252. https://doi.org/10.1016/j.juro.2011.09.032

    Article  PubMed  Google Scholar 

  12. Zhenzhu L, Lu L, Zhenzhi L, Xuzhi L, Yizhi L, Gangxian F, Henglu W, Jinke D, Qingbo W, Pengfei L, Meng L (2020) Feasibility study of the low-cost motion tracking system for assessing endoscope holding skills. World Neurosurg 140:312–319. https://doi.org/10.1016/j.wneu.2020.04.191

    Article  PubMed  Google Scholar 

  13. Mansoor SM, Våpenstad C, Mårvik R, Glomsaker T, Bliksøen M (2020) Construct validity of eoSim–a low-cost and portable laparoscopic simulator. Minim Invasive Ther Allied Technol 29:261–268. https://doi.org/10.1080/13645706.2019.1638411

    Article  PubMed  Google Scholar 

  14. Hung AJ, Chen J, Jarc A, Hatcher D, Djaladat H, Gill IS (2018) Development and validation of objective performance metrics for robot-assisted radical prostatectomy: a pilot study. J Urol 199:296–304. https://doi.org/10.1016/j.juro.2017.07.081

    Article  PubMed  Google Scholar 

  15. Oropesa I, Sánchez-González P, Chmarra MK, Lamata P, Fernández Á, Sánchez-Margallo JA, Jansen FW, Dankelman J, Sánchez-Margallo FM, Gómez EJ (2013) EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg Endosc 27:1029–1039. https://doi.org/10.1007/s00464-012-2513-z

    Article  PubMed  Google Scholar 

  16. Azari DP, Miller BL, Le BV, Greenberg CC, Radwin RG (2020) Quantifying surgeon maneuevers across experience levels through marker-less hand motion kinematics of simulated surgical tasks. Appl Ergon 87:103136. https://doi.org/10.1016/j.apergo.2020.103136

    Article  PubMed  Google Scholar 

  17. Zheng Y, Leonard G, Tellez J, Zeh H, Fey AM (2021) Identifying kinematic markers associated with intraoperative stress during surgical training tasks. In: International symposium on medical robotics (ISMR), pp 1–7

  18. Schmidt MW, Kowalewski KF, Schmidt ML, Wennberg E, Garrow CR, Paik S, Benner L, Schijven MP, Müller-Stich BP, Nickel F (2019) The heidelberg VR score: development and validation of a composite score for laparoscopic virtual reality training. Surg Endosc 33:2093–2103. https://doi.org/10.1007/s00464-018-6480-x

    Article  PubMed  Google Scholar 

  19. Ershad M, Rege R, Majewicz Fey A (2019) Automatic and near real-time stylistic behavior assessment in robotic surgery. Int J Comput Assist Radiol Surg 14:635–643. https://doi.org/10.1007/s11548-019-01920-6

    Article  CAS  PubMed  Google Scholar 

  20. Nisky I, Okamura AM, Hsieh MH (2014) Effects of robotic manipulators on movements of novices and surgeons. Surg Endosc 28:2145–2158. https://doi.org/10.1007/s00464-014-3446-5

    Article  PubMed  PubMed Central  Google Scholar 

  21. Yamaguchi T, Nakamura R (2018) Laparoscopic training using a quantitative assessment and instructional system. Int J Comput Assist Radiol Surg 13:1453–1461. https://doi.org/10.1007/s11548-018-1771-1

    Article  CAS  PubMed  Google Scholar 

  22. Kitaguchi D, Takeshita N, Matsuzaki H, Oda T, Watanabe M, Mori K, Kobayashi E, Ito M (2020) Automated laparoscopic colorectal surgery workflow recognition using artificial intelligence: experimental research. Int J Surg 79:88–94. https://doi.org/10.1016/j.ijsu.2020.05.015

    Article  PubMed  Google Scholar 

  23. Hofstad EF, Våpenstad C, Chmarra MK, Langø T, Kuhry E, Mårvik R (2013) A study of psychomotor skills in minimally invasive surgery: what differentiates expert and nonexpert performance. Surg Endosc 27:854–863. https://doi.org/10.1007/s00464-012-2524-9

    Article  PubMed  Google Scholar 

  24. Escamirosa FP, Flores RMO, García IO, Vidal CRZ, Martínez AM (2015) Face, content, and construct validity of the EndoViS training system for objective assessment of psychomotor skills of laparoscopic surgeons. Surg Endosc 29:3392–3403. https://doi.org/10.1007/s00464-014-4032-6

    Article  PubMed  Google Scholar 

  25. Vassiliou MC, Dunkin BJ, Marks JM, Fried GM (2010) FLS and FES: comprehensive models of training and assessment. Surg Clin North Am 90:535–558. https://doi.org/10.1016/j.suc.2010.02.012

    Article  PubMed  Google Scholar 

  26. Aghazadeh F, Zheng B, Tavakoli M, Rouhani H (2023) Motion smoothness-based assessment of surgical expertise: the importance of selecting proper metrics. Sensors 23:3146. https://doi.org/10.3390/s23063146

    Article  PubMed  PubMed Central  Google Scholar 

  27. Aghazadeh F, Arjmand N, Nasrabadi AM (2020) Coupled artificial neural networks to estimate 3D whole-body posture, lumbosacral moments, and spinal loads during load-handling activities. J Biomech 102:109332. https://doi.org/10.1016/j.jbiomech.2019.109332

    Article  CAS  PubMed  Google Scholar 

  28. Müller M (2007) Dynamic time warping. Inf Retr Music motion, pp 69–84

  29. Fathian R, Phan S, Ho C, Rouhani H (2023) Face touch monitoring using an instrumented wristband using dynamic time warping and k-nearest neighbours. PLoS ONE 18:e0281778. https://doi.org/10.1371/journal.pone.0281778

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  30. Fard MJ, Ameri S, Darin Ellis R, Chinnam RB, Pandya AK, Klein MD (2018) Automated robot-assisted surgical skill evaluation: predictive analytics approach. Int. J. Med. Robot 14:e1850. https://doi.org/10.1002/rcs.1850

    Article  Google Scholar 

  31. Schaverien MV (2010) Development of expertise in surgical training. J Surg Educ 67:37–43. https://doi.org/10.1016/j.jsurg.2009.11.002

    Article  PubMed  Google Scholar 

  32. Chmarra MK, Kolkman W, Jansen FW, Grimbergen CA, Dankelman J (2007) The influence of experience and camera holding on laparoscopic instrument movements measured with the TrEndo tracking system. Surg Endosc 21:2069–2075. https://doi.org/10.1007/s00464-007-9298-5

    Article  CAS  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the study participants for their contribution to this study.

Funding

This work was financially supported by the Alberta Minister of Jobs, Economy and Innovation, Major Innovation Fund—Autonomous Systems Initiative (MIF01 T4 P1).

Author information

Authors and Affiliations

  1. Department of Mechanical Engineering, 10-390 Donadeo Innovation Centre for Engineering, University of Alberta, 9211-116 Street NW, Edmonton, AB, T6G 1H9, Canada

    Farzad Aghazadeh & Hossein Rouhani

  2. Department of Surgery, University of Alberta, Edmonton, AB, Canada

    Bin Zheng

  3. Department of Electrical and Computer Engineering, University of Alberta, Edmonton, AB, Canada

    Mahdi Tavakoli

Authors
  1. Farzad Aghazadeh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bin Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mahdi Tavakoli
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hossein Rouhani
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hossein Rouhani.

Ethics declarations

Conflict of interest

None.

Ethical approval

The Research Ethics Review Board of the University of Alberta approved the study protocol (Study ID: Pro00114163).

Informed consent

A written consent form was signed by all the participants prior to the tests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aghazadeh, F., Zheng, B., Tavakoli, M. et al. Surgical tooltip motion metrics assessment using virtual marker: an objective approach to skill assessment for minimally invasive surgery. Int J CARS 18, 2191–2202 (2023). https://doi.org/10.1007/s11548-023-03007-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-023-03007-9

Keywords

Search

Navigation