Skip to main content
SpringerLink
Log in

Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

   Virtual reality simulator technology together with novel metrics could advance our understanding of expert neurosurgical performance and modify and improve resident training and assessment. This pilot study introduces innovative metrics that can be measured by the state-of-the-art simulator to assess performance. Such metrics cannot be measured in an operating room and have not been used previously to assess performance.

Methods

   Three sets of performance metrics were assessed utilizing the NeuroTouch platform in six scenarios with simulated brain tumors having different visual and tactile characteristics. Tier 1 metrics included percentage of brain tumor resected and volume of simulated “normal” brain tissue removed. Tier 2 metrics included instrument tip path length, time taken to resect the brain tumor, pedal activation frequency, and sum of applied forces. Tier 3 metrics included sum of forces applied to different tumor regions and the force bandwidth derived from the force histogram.

Results

   The results outlined are from a novice resident in the second year of training and an expert neurosurgeon. The three tiers of metrics obtained from the NeuroTouch simulator do encompass the wide variability of technical performance observed during novice/expert resections of simulated brain tumors and can be employed to quantify the safety, quality, and efficiency of technical performance during simulated brain tumor resection. Tier 3 metrics derived from force pyramids and force histograms may be particularly useful in assessing simulated brain tumor resections.

Conclusion

   Our pilot study demonstrates that the safety, quality, and efficiency of novice and expert operators can be measured using metrics derived from the NeuroTouch platform, helping to understand how specific operator performance is dependent on both psychomotor ability and cognitive input during multiple virtual reality brain tumor resections.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Abbreviations

NRC:

National Research Council

VR:

Virtual reality

PGY:

Postgraduate year

cc:

Cubic centimeters

3D:

Three-dimensional

References

  1. Brydges R, Kurahashi A, Brümmer V, Satterthwaite L, Classen R, Dubrowski A (2008) Developing criteria for proficiency-based training of surgical technical skills using simulation: changes in performances as a function of training year. J Am Coll Surg 206:205–211. doi:10.1016/j.jamcollsurg.2007.07.045

    Article  PubMed  Google Scholar 

  2. Aggarwal R, Grantcharov TR, Eriksen JR, Blirup D, Kristiansen VB, Funch-Jensen P, Darzi A (2006) An evidence-based virtual reality training program for novice laparoscopic surgeons. Ann Surg 244:310–314. doi:10.1097/01.sla.0000218094.92650.44

    Article  PubMed Central  PubMed  Google Scholar 

  3. Ericsson KA, Charness N (1994) Expert performance. Am Psychol 49:725–747

    Article  Google Scholar 

  4. Tsuda S, Scott D, Doyle J, Jones DB (2009) Surgical skills training and simulation. Curr Probl Surg 46:271–370. doi:10.1067/j.cpsurg.2008.12.003

    Article  PubMed  Google Scholar 

  5. Jaffer A, Bednarz B, Challacombe B, Sriprasad S (2009) The assessment of surgical competency in the UK. Int J Surg 7:12–15. doi:10.1016/j.ijsu.2008.10.006

    Article  CAS  PubMed  Google Scholar 

  6. Reznick RK, MacRae H (2006) Teaching surgical skills—changes in the wind. N Eng J Med 355:2664–2669. doi:10.1056/NEJMra054785

    Article  CAS  Google Scholar 

  7. Brightwell A, Grant J (2013) Competency-based training: who benefits? Postgrad Med J 89:107–110. doi:10.1136/postgradmedj-2012-130881

    Article  PubMed  Google Scholar 

  8. Bhatti NI, Cummings CW (2007) Competency in surgical residency training: defining and raising the bar. Acad Med 82:569–573

    Article  PubMed  Google Scholar 

  9. Gélinas-Phaneuf N, Del Maestro RF (2013) Surgical expertise in neurosurgery integrating theory into practice. Neurosurgery 73(Suppl. 1):30–38. doi:10.1227/NEU.0000000000000115

    Article  PubMed  Google Scholar 

  10. Gaba DM, DeAnda A (1998) A comprehensive anesthesia simulation environment: re-creating the operating room for research and training. Anesthesiology 69:387–394

    Article  Google Scholar 

  11. Chung J, Sackier J (1998) A method of objectively evaluating improvements in laparoscopic skills. Surg Endosc 12:1111–1116

    Article  CAS  PubMed  Google Scholar 

  12. Derossis AM, Bothwell J, Sigman HH, Fried GM (1998) The effect of practice on performance in a laparoscopic simulator. Surg Endosc 12:1117–1120

    Article  CAS  PubMed  Google Scholar 

  13. Delp SL, Loan JP, Hoy MG, Zajac FE, Topp EL, Rosen JM (1990) An interactive graphics-based model of the lower extremity to study orthopaedic surgical procedures. IEEE Trans Biomed Eng 37:757–767

    Article  CAS  PubMed  Google Scholar 

  14. Satava RM (1993) Virtual reality surgical simulator. Surg Endosc 7:203–205

    Article  CAS  PubMed  Google Scholar 

  15. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA 3rd, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. doi:10.1016/j.amjsurg.2006.06.050

    Article  PubMed  Google Scholar 

  16. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM, Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. doi:10.1097/01.SLA.0000028969.51489

    Article  PubMed Central  PubMed  Google Scholar 

  17. Larsen CR, Soerensen JL, Grantcharov TP, Dalsgaard T, Schouenborg L, Ottosen C, Schroeder TV, Ottesen BS (2009) Effect of virtual reality training on laparoscopic surgery: randomised controlled trial. BMJ 338:b1802. doi:10.1136/bmj.b1802

    Article  PubMed Central  PubMed  Google Scholar 

  18. Phillips NI, John NW (2000) Web-based surgical simulation for ventricular catheterization. Neurosurgery 46:933–937

    CAS  PubMed  Google Scholar 

  19. Ferroli P, Tringali G, Acerbi F, Schiariti M, Broggi M, Aquino D, Broggi G (2013) Advanced 3-dimensional planning in neurosurgery. Neurosurgery 72(Suppl 1):54–62. doi:10.1227/NEU.0b013e3182748ee8

    Article  PubMed  Google Scholar 

  20. Neubauer A, Wolfsberger S (2013) Virtual endoscopy in neurosurgery: a review. Neurosurgery 72(Suppl 1):97–106. doi:10.1227/NEU.0b013e31827393c9

    Article  PubMed  Google Scholar 

  21. Mitha AP, Almekhlafi MA, Janjua MJ, Albuquerque FC, McDougall CG (2013) Simulation and augmented reality in endovascular neurosurgery: lessons from aviation. Neurosurgery 72(Suppl 1):107–114. doi:10.1227/NEU.0b013e31827981fd

    Article  PubMed  Google Scholar 

  22. Rousseau G, Bailes J, del Maestro RF, Cabral A, Choudhury N, Comas O, Debergue P, De Luca G, Hovdebo J, Jiang D, Laroche D, Neubauer A, Pazos V, Thibault T, DiRaddo R (2013) The development of a virtual simulator for training of neurosurgeons to perform and to perfect endoscopic endonasal transphenoidal surgery. Neurosurgery 73(Suppl. 1):85–93. doi:10.1227/NEU.0000000000000112

    Article  Google Scholar 

  23. Kockro RA, Serra L, Tseng-Tsai Y, Yih-Yian Chan CS, Gim-Guan C, Lee E, Hoe LY, Hern N, Nowinski WL (2000) Planning and simulation of neurosurgery in a virtual reality environment. Neurosurgery 46:118–137

    Article  CAS  PubMed  Google Scholar 

  24. Luciano C, Banerjee P, Lemole G, Charbel F (2006) Second generation haptic ventriculostomy simulator using the immersive touch system. Stud Health Technol Inform 119:343–348

  25. Banerjee PP, Luciano CJ, Lemole GM Jr, Charbel FT, Oh MY (2007) Accuracy of ventriculostomy catheter placement using a head- and hand-tracked high-resolution virtual reality simulator with haptic feedback. J Neurosurg 107:515–521. doi:10.3171/JNS-07/09/0515

    Article  PubMed  Google Scholar 

  26. Luciano CJ, Banerjee PP, Bellotte B, Oh GM, Lemole M, Jr., Charbel FT, Roitberg B (2011) Learning retention of thoracic pedicle screw placement using a high-resolution augmented reality simulator with haptic feedback. Neurosurgery 69(1 Suppl Operative):ons14–19. doi:10.1227/NEU.0b013e31821954ed

  27. Luciano CJ, Banerjee PP, Florea L, Dawe G (2005) Design of the ImmersiveTouch\(^{TM}\): a high-performance haptic augmented virtual reality system. In: 11th International conference on human-computer interaction, Las Vegas, NV

  28. Delorme S, Laroche D, Diraddo R, F. Del Maestro R (2012) NeuroTouch: a Physics-based virtual simulator for cranial microneurosurgery training. Neurosurgery 71(1 Suppl Operative):ons 32–42. doi:10.1227/NEU.0b013e318249c744

  29. Choudhury N, Gelinas-Phaneuf N, Delorme S, Del Maestro R (2013) Fundamentals of Neurosurgery: virtual reality tasks for training and evaluation of technical skills. World Neurosurg 80:E9–E19. doi:10.1016/j.wneu.2012.08.022

  30. Gélinas-Phaneuf N, Choudhury N, Al-Habib AR, Cabral A, Nadeau E, Mora V, Debergue Pazos VP, Diraddo R, Del Maestro RF (2013) Assessing performance in brain tumor resection using a novel virtual reality simulator. Int J CARS. doi:10.1007/s11548-013-0905-8

  31. Wheatstone C (1838) Contributions to the physiology of vision.-Part the first. On some remarkable, and hitherto unobserved, phenomena of binocular vision. Philos Trans R Soc Lond 128:371–394

    Article  Google Scholar 

  32. Kohn LT, Corrigan J, Donaldson MS (2000) To err is human: building a safer health care system. National Academy Press, Washington

    Google Scholar 

  33. Gray JD (1996) Global rating scale in residency education. Acad Med 71:555–563

    Article  Google Scholar 

  34. Turnbull J, Gray J, MacFadyen J (1998) Improving in training evaluation programs. J Gen Intern Med 13:317–323. doi:10.1046/j.1525-1497.1998.00097.x

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  35. Katz R, Hoznek A, Salomon L, Antiphon P, de la Taille A, Abbou CC (2005) Skill assessment of urological laparoscopic surgeons: can criterion levels of surgical performance be determined using the pelvic box trainer? Eur Urol 47:482–487. doi:10.1016/j.eururo.2004.12.007

    Article  PubMed  Google Scholar 

  36. de Vries EN, Ramrattan MA, Smorenburg SM, Gouma DJ, Boermeester MA (2008) The incidence and nature of in-hospital adverse events: a systemic review. Qual Saf Health Care 17:216–23. doi:10.1136/qshc.2007.023622

    Article  PubMed Central  PubMed  Google Scholar 

  37. Gawande AA, Thomas EJ, Zinner MJ, Brennan TA (1999) The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 126:66–75

    Article  CAS  PubMed  Google Scholar 

  38. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJ (2013) Surgical skills and complication rates after bariatric surgery. N Engl J Med 369:1434–1442. doi:10.1056/NEJMsa1300625

    Article  CAS  PubMed  Google Scholar 

  39. Kockro RA (2013) Neurosurgery simulators-beyond the experiment. World Neurosurg. 80:E101–E102. doi:10.1016/j.wneu.2013.02.017

Download references

Acknowledgments

This work was supported by The Di Giovanni Foundation, the Montreal English School Board, the B-Strong Foundation, the Colannini Foundation, and the Montreal Neurological Institute and Hospital. Dr. R. F. Del Maestro is the William Feindel Emeritus Professor in Neuro-Oncology at McGill University. Dr. Azarnoush held the Postdoctoral Neuro-Oncology Fellowship from the Montreal Neurological Institute and Hospital. Dr. Gélinas-Phaneuf was funded by a generous contribution from the Harold and Audrey Fisher Brain Tumour Research Award.

Author information

Authors and Affiliations

  1. Neurosurgical Simulation Research Center, Department of Neurosurgery and Neurology, 2nd Floor, Montreal Neurological Institute and Hospital, McGill University, 3801 University, Suite W201, Montreal, QC, H3A 2B4, Canada

    Hamed Azarnoush, Gmaan Alzhrani, Alexander Winkler-Schwartz, Fahad Alotaibi, Nicholas Gelinas-Phaneuf, Jawad Fares & Rolando F. Del Maestro

  2. National Research Council Canada, Boucherville, QC, Canada

    Valérie Pazos, Nusrat Choudhury & Robert DiRaddo

  3. Department of Biomedical Engineering, Tehran Polytechnic, Tehran, Iran

    Hamed Azarnoush

  4. National Neuroscience Institute, King Fahad Medical City, Riyadh, Saudi Arabia

    Gmaan Alzhrani & Fahad Alotaibi

Authors
  1. Hamed Azarnoush
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gmaan Alzhrani
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alexander Winkler-Schwartz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fahad Alotaibi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Nicholas Gelinas-Phaneuf
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Valérie Pazos
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Nusrat Choudhury
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jawad Fares
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Robert DiRaddo
    View author publications

    You can also search for this author in PubMed Google Scholar

  10. Rolando F. Del Maestro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hamed Azarnoush.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (wmv 4794 KB)

Supplementary material 2 (avi 7285 KB)

Supplementary material 3 (avi 5134 KB)

Supplementary material 4 (doc 10 KB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Azarnoush, H., Alzhrani, G., Winkler-Schwartz, A. et al. Neurosurgical virtual reality simulation metrics to assess psychomotor skills during brain tumor resection. Int J CARS 10, 603–618 (2015). https://doi.org/10.1007/s11548-014-1091-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-014-1091-z

Keywords

Search

Navigation