Abstract
Current surgical skill assessment mainly relies on evaluations by senior surgeons, a tedious process influenced by subjectivity. The contradiction between a growing number of surgical techniques and the duty-hour limits for residents leads to an increasing need for effective surgical skill assessment. In this paper, we explore an automatic surgical skill assessment method by tracking and analyzing the surgery trajectories in a new dataset of endoscopic cadaveric trans-nasal sinus surgery videos. The tracking is performed by combining the deep convolutional neural network based segmentation and the dense optical flow algorithm. Then the heat maps and motion metrics of the tip trajectories are extracted and analyzed. The proposed method has been tested in 10 endoscopic videos of sinus surgery performed by 4 expert and 5 novice surgeons, showing the potential for the automatic surgical skill assessment.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Aggarwal, R., Moorthy, K., Darzi, A.: Laparoscopic skills training and assessment. Br. J. Surg. 91(12), 1549–1558 (2004)
Bouget, D., Allan, M., Stoyanov, D., Jannin, P.: Vision-based and marker-less surgical tool detection and tracking: a review of the literature. Med. Image Anal. 35, 633–654 (2017)
Carter, B.N.: The fruition of halsted’s concept of surgical training. Surgery 32(3), 518–527 (1952)
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Farnebäck, G.: Two-frame motion estimation based on polynomial expansion. In: Bigun, J., Gustavsson, T. (eds.) SCIA 2003. LNCS, vol. 2749, pp. 363–370. Springer, Heidelberg (2003). https://doi.org/10.1007/3-540-45103-X_50
Ismail Fawaz, H., Forestier, G., Weber, J., Idoumghar, L., Muller, P.-A.: Evaluating surgical skills from kinematic data using convolutional neural networks. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 214–221. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_25
Jin, A., et al.: Tool detection and operative skill assessment in surgical videos using region-based convolutional neural networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 691–699. IEEE (2018)
Lahanas, V., Georgiou, E., Loukas, C.: Surgical simulation training systems: box trainers, virtual reality and augmented reality simulators. Int. J. Adv. Robot. Autom. 1(2), 1–9 (2016)
Liu, D., Jiang, T.: Deep reinforcement learning for surgical gesture segmentation and classification. In: Frangi, A.F., Schnabel, J.A., Davatzikos, C., Alberola-López, C., Fichtinger, G. (eds.) MICCAI 2018. LNCS, vol. 11073, pp. 247–255. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00937-3_29
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Noble, W.S.: How does multiple testing correction work? Nat. Biotechnol. 27(12), 1135 (2009)
Oropesa, I., et al.: EVA: laparoscopic instrument tracking based on endoscopic video analysis for psychomotor skills assessment. Surg. Endosc. 27(3), 1029–1039 (2013)
Reiley, C.E., Lin, H.C., Yuh, D.D., Hager, G.D.: Review of methods for objective surgical skill evaluation. Surg. Endosc. 25(2), 356–366 (2011)
Shvets, A.A., Rakhlin, A., Kalinin, A.A., Iglovikov, V.I.: Automatic instrument segmentation in robot-assisted surgery using deep learning. In: 2018 17th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 624–628. IEEE (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lin, S., Qin, F., Bly, R.A., Moe, K.S., Hannaford, B. (2020). Automatic Sinus Surgery Skill Assessment Based on Instrument Segmentation and Tracking in Endoscopic Video. In: Li, Q., Leahy, R., Dong, B., Li, X. (eds) Multiscale Multimodal Medical Imaging. MMMI 2019. Lecture Notes in Computer Science(), vol 11977. Springer, Cham. https://doi.org/10.1007/978-3-030-37969-8_12
Download citation
DOI: https://doi.org/10.1007/978-3-030-37969-8_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-37968-1
Online ISBN: 978-3-030-37969-8
eBook Packages: Computer ScienceComputer Science (R0)
