Skip to main content
Springer Nature Link
Log in

Assessment of Core Surgical Skills Using a Mixed Reality Headset – The MoTOR Study

  • Education & Training
  • Published:
Journal of Medical Systems Aims and scope Submit manuscript

Abstract

Introduction

Surgical skill assessment utilises direct observation and feedback by an expert which is potentially subjective, therefore obtaining objective data for hand and eye tracking is essential. Our aim was to evaluate a wearable mixed reality (MR) headset in these domains.

Methods

Participants with differing levels of surgical expertise [novice (N), intermediate (I) & expert (E)] performed 4 simulated surgical tasks; 2 general dexterity (tasks 1&2) and 2 surgical skills (tasks 3&4) wearing the MR headset capturing their hand and eye movements (median & range). Metrics included hand path length and the speed of each index or thumb tip. Gaze data were also captured. Participant demographics, prior expertise and current experience were captured with an electronic survey. Data were analysed with a Shapiro-Wilk test or ANOVA as appropriate. A p-value of < 0.05 was significant.

Results

Thirty-six participants were analysed (N = 18, I = 8, E = 8). Tasks 1&2 revealed 2 speed outcomes (left index and left-hand speed) which were significant. For tasks 3&4, various outcomes were significant: path length for left hand (N:45 cm vs. I:31 cm vs. E:27 cm, p = 0.03) and right hand (N:48 cm vs. I:29 cm vs. E:28 cm, p = 0.01) and total time (N:456s vs. I:292 vs. E: 245, p = 0.0002). With left-hand-tying, average path length (N:61 cm vs. I:39 vs. E:36, p = 0.04), average speed (N:11 cm/s vs. I:23 vs. E:24, p = 0.03), and total time (N:156s vs. I:43 vs. E:37, p = 0.003) were significant. The gaze-tracking was not statistically significant.

Conclusion

The MR headset can be utilised as a valid tool for surgical performance assessment. Outcomes including path length and speed can be valuable metrics captured by the MR Headset during the task completion for detecting surgical proficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Cameron JL. William Stewart Halsted. Our surgical heritage. Ann Surg. 1997;225(5):445–58.

    Article  CAS  Google Scholar 

  2. Hiemstra E, Kolkman W, Wolterbeek R, Trimbos B, Jansen FW. Value of an objective assessment tool in the operating room. Can J Surg J Can De Chir. 2011;54(2):116–22.

    Article  Google Scholar 

  3. Hove PDv, Tuijthof GJM, Verdaasdonk EGG, Stassen LPS, Dankelman J. Objective assessment of technical surgical skills. Brit J Surg. 2010;97(7):972–87.

    Article  Google Scholar 

  4. Vedula SS, Ishii M, Hager GD. Objective Assessment of Surgical Technical Skill and Competency in the Operating Room. Annu Rev Biomed Eng. 2017;19(1):301–25.

    Article  CAS  Google Scholar 

  5. Shackelford S, Bowyer M. Modern Metrics for Evaluating Surgical Technical Skills. Curr Surg Reports. 2017;5(10):24.

    Article  Google Scholar 

  6. Mason JD, Ansell J, Warren N, Torkington J. Is motion analysis a valid tool for assessing laparoscopic skill? Surg Endosc. 2012;27(5):1468–77.

    Article  Google Scholar 

  7. Datta V, Mackay S, Mandalia M, Darzi A. The use of electromagnetic motion tracking analysis to objectively measure open surgical skill in the laboratory-based model 1 1No competing interests declared. J Am Coll Surgeons. 2001;193(5):479–85.

    Article  CAS  Google Scholar 

  8. Nasr A, Carrillo B, Gerstle JT, Azzie G. Motion analysis in the pediatric laparoscopic surgery (PLS) simulator: validation and potential use in teaching and assessing surgical skills. J Pediatr Surg. 2014;49(5):791–4.

    Article  Google Scholar 

  9. Smith SGT, Torkington J, Brown TJ, Taffinder NJ, Darzi A. Motion analysis: A tool for assessing laparoscopic dexterity in the performance of a laboratory-based laparoscopic cholecystectomy. Surg Endosc. 2001;16(4):640–5.

    Article  Google Scholar 

  10. Trudeau MON, Nasr A, Carrillo B, Gerstle JT, Azzie G. Construct validity and educational role for motion analysis in a laparoscopic trainer. Surg Endosc. 2014;29(9):2491–5.

    Article  Google Scholar 

  11. Sickle KRV, Iii DAM, Gallagher AG, Smith CD. Construct validation of the ProMIS simulator using a novel laparoscopic suturing task. Surgical Endoscopy And Other Interventional Techniques. 2005;19(9):1227–31.

    Article  Google Scholar 

  12. Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, et al. Systematic review of skills transfer after surgical simulation-based training. Brit J Surg. 2014;101(9):1063–76.

    Article  CAS  Google Scholar 

  13. Microsoft. Getting around the HoloLens 2 Microsoft Mixed Reality Documentation: Microsoft; 2019. Available from: https://docs.microsoft.com/en-us/hololens/hololens2-basic-usage.

  14. Bann SD, Khan MS, Darzi AW. Measurement of Surgical Dexterity Using Motion Analysis of Simple Bench Tasks. World J Surg. 2003;27(4):390–4.

    Article  Google Scholar 

  15. Atesok K, Satava RM, Marsh JL, Hurwitz SR. Measuring Surgical Skills in Simulation-based Training. J Am Acad Orthop Surg. 2017;25(10):665–72.

    Article  Google Scholar 

  16. Howells NR, Brinsden MD, Gill RS, Carr AJ, Rees JL. Motion Analysis: A Validated Method for Showing Skill Levels in Arthroscopy. Arthrosc J Arthrosc Relat Surg. 2008;24(3):335–42.

    Article  Google Scholar 

  17. Wilson M, McGrath J, Vine S, Brewer J, Defriend D, Masters R. Psychomotor control in a virtual laparoscopic surgery training environment: gaze control parameters differentiate novices from experts. Surg Endosc. 2010;24(10):2458–64.

    Article  Google Scholar 

  18. Erridge S, Ashraf H, Purkayastha S, Darzi A, Sodergren MH. Comparison of gaze behaviour of trainee and experienced surgeons during laparoscopic gastric bypass. Brit J Surg. 2017;105(3):287–94.

    Article  Google Scholar 

  19. Brydges R, Classen R, Larmer J, Xeroulis G, Dubrowski A. Computer-assisted assessment of one-handed knot tying skills performed within various contexts: a construct validity study. Am J Surg. 2006;192(1):109–13.

    Article  Google Scholar 

Download references

Acknowledgements

This study as it was simulation-based educational research was not preregistered in an independent institutional registry.

Funding

This study received no funding.

Author information

Authors and Affiliations

  1. Department of Paediatric Surgery & Monash Children’s Simulation, Monash Children’s Hospital, 246 Clayton Road, Clayton, 3168, Melbourne, Australia

    John Valles, Maurizio Pacilli & Ramesh M. Nataraja

  2. Virtual and Augmented Reality Services Unit, Monash eSolutions, Monash University, Melbourne, VIC, Australia

    Taiqing Zhang & Paul McIntosh

  3. Human Centred Computing, Faculty of Information Technology, Monash University, Melbourne, VIC, Australia

    Paul McIntosh

  4. Departments of Paediatrics & Surgery, School of Clinical Sciences, Faculty of Medicine, Nursing and Health Sciences, Monash University, Melbourne, Australia

    John Valles, Maurizio Pacilli & Ramesh M. Nataraja

Authors
  1. John Valles
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Taiqing Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Paul McIntosh
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Maurizio Pacilli
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Ramesh M. Nataraja
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ramesh M. Nataraja.

Ethics declarations

Presentation

This abstract was presented at the Pacific Association of Pediatric Surgeons Virtual Congress November 2021.

Disclosures

There are no conflicts of interests or financial ties for any of the parties involved in this study. This includes the whole authorship of John Valles, Taiqing Zhang, Paul McIntosh, Maurizio Pacilli, and Ramesh M. Nataraja.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Education & Training

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Valles, J., Zhang, T., McIntosh, P. et al. Assessment of Core Surgical Skills Using a Mixed Reality Headset – The MoTOR Study. J Med Syst 46, 102 (2022). https://doi.org/10.1007/s10916-022-01891-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10916-022-01891-3

Keywords