Skip to main content
Springer Nature Link
Log in

An explainable machine learning method for assessing surgical skill in liposuction surgery

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Surgical skill assessment has received growing interest in surgery training and quality control due to its essential role in competency assessment and trainee feedback. However, the current assessment methods rarely provide corresponding feedback guidance while giving ability evaluation. We aim to validate an explainable surgical skill assessment method that automatically evaluates the trainee performance of liposuction surgery and provides visual postoperative and real-time feedback.

Methods

In this study, machine learning using a model-agnostic interpretable method based on stroke segmentation was introduced to objectively evaluate surgical skills. We evaluated the method on liposuction surgery datasets that consisted of motion and force data for classification tasks.

Results

Our classifier achieved optimistic accuracy in clinical and imitation liposuction surgery models, ranging from 89 to 94%. With the help of SHapley Additive exPlanations (SHAP), we deeply explore the potential rules of liposuction operation between surgeons with variant experiences and provide real-time feedback based on the ML model to surgeons with undesirable skills.

Conclusion

Our results demonstrate the strong abilities of explainable machine learning methods in objective surgical skill assessment. We believe that the machine learning model based on interpretive methods proposed in this article can improve the evaluation and training of liposuction surgery and provide objective assessment and training guidance for other surgeries.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

References

  1. Birkmeyer JD, Finks JF, O’Reilly A, Oerline M, Carlin AM, Nunn AR, Dimick J, Banerjee M, Birkmeyer NJO (2013) Surgical Skill and complication rates after bariatric surgery. New Engl J Med 369(15):1434–42. https://doi.org/10.1056/NEJMsa1300625

    Article  PubMed  CAS  Google Scholar 

  2. Nathan M, Karamichalis JM, Liu H, Emani S, Baird C, Pigula F, Colan S, Thiagarajan RR, Bacha EA, del Nido P (2012) Surgical technical performance scores are predictors of late mortality and unplanned reinterventions in infants after cardiac surgery. J Thoracic Cardiovasc Surg 144(5):1095. https://doi.org/10.1016/j.jtcvs.2012.07.081

    Article  Google Scholar 

  3. Castillo-Segura P, Fernandez-Panadero C, Alario-Hoyos C, Munoz-Merino PJ, Delgado KC (2021) Objective and automated assessment of surgical technical skills with IoT systems: a systematic literature review. Artif Intell Med 112:102007. https://doi.org/10.1016/j.artmed.2020.102007

    Article  PubMed  Google Scholar 

  4. Ericsson KA (2004) Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Acad Med 79(10):S70–S81

    Article  PubMed  Google Scholar 

  5. Forestier G, Riffaud L, Petitjean F, Henaux P-L, Jannin P (2018) Surgical skills: Can learning curves be computed from recordings of surgical activities? Int J Comput Assist Radiol Surg 13(5):629–36. https://doi.org/10.1007/s11548-018-1713-y

    Article  PubMed  Google Scholar 

  6. Forestier G, Petitjean F, Senin P, Despinoy F, Huaulme A, Fawaz HI, Weber J, Idoumghar L, Muller P-A, Jannin P (2018) Surgical motion analysis using discriminative interpretable patterns. Artif Intell Med 91:3–11. https://doi.org/10.1016/j.artmed.2018.08.002

    Article  PubMed  Google Scholar 

  7. Forestier G, Petitjean F, Senin P, Despinoy F, Jannin P (2017) Discovering discriminative and interpretable patterns for surgical motion analysis. In: ten Teije Annette, Popow Christian, Holmes John H, Sacchi Lucia (eds) Artificial intelligence in medicine. Springer International Publishing, Cham

    Google Scholar 

  8. Ismail Fawaz H, Forestier G, Weber J, Idoumghar L, Muller P-A (2019) Accurate and interpretable evaluation of surgical skills from kinematic data using fully convolutional neural networks. Int J Comput Assist Radiol Surg 14(9):1611–7. https://doi.org/10.1007/s11548-019-02039-4

    Article  PubMed  Google Scholar 

  9. Uemura M, Tomikawa M, Miao T, Souzaki R, Ieiri S, Akahoshi T, Lefor AK, Hashizume M (2018) Feasibility of an AI-based measure of the hand motions of expert and novice surgeons. Comput Math Method Med 2018:1–6. https://doi.org/10.1155/2018/9873273

    Article  Google Scholar 

  10. Lundberg SM, Lee S-I, (2017). A unified approach to interpreting model predictions. In: 31st annual conference on neural information processing Systems (NIPS); 2017 Dec pp 04–09; Long Beach, CA2017

  11. Ribeiro MT, Singh S, Guestrin C, Assoc Comp M, (2016)."Why Should I Trust You?" explaining the predictions of any classifier. In: 22nd ACM SIGKDD international conference on knowledge discovery and data mining (KDD); 2016 Aug pp 13–17; San Francisco, CA2016

  12. Strumbelj E, Kononenko I (2014) Explaining prediction models and individual predictions with feature contributions. Knowl Inform Syst 41(3):647–65. https://doi.org/10.1007/s10115-013-0679-x

    Article  Google Scholar 

  13. Illouz YG (1983) Body contouring by lipolysis: a 5-year experience with over 3000 cases. Plastic Reconstr Surg 72(5):591–7. https://doi.org/10.1097/00006534-198311000-00001

    Article  CAS  Google Scholar 

  14. Matarasso A, Courtiss EH (1991) Suction mammaplasty: the use of suction lipectomy to reduce large breasts. Plastic Reconstr Surg 87(4):709–17. https://doi.org/10.1097/00006534-199104000-00016

    Article  CAS  Google Scholar 

  15. Mladick RA (1990) Lipoplasty of the calves and ankles. Plastic Reconstr Surg 86(1):84–93. https://doi.org/10.1097/00006534-199007000-00013

    Article  CAS  Google Scholar 

  16. Dixit VV, Wagh MS (2013) Unfavourable outcomes of liposuction and their management. Indian J Plast Surg 46(2):377–92. https://doi.org/10.4103/0970-0358.118617

    Article  PubMed  PubMed Central  Google Scholar 

  17. Liu Y, Yibulayimu S, Sun Z, Wang Y, Wang Y, Li F, (2021) Design of novel adipose tissue mimicking phantom material for liposuction training. In: 14th international congress on image and signal processing, biomedical engineering and informatics (CISP-BMEI); 2021 pp 23–25 Oct. 2021

  18. Ahmidi N, Poddar P, Jones JD, Vedula SS, Ishii L, Hager GD, Ishii M (2015) Automated objective surgical skill assessment in the operating room from unstructured tool motion in septoplasty. Int J Comput Assist Radiol Surg 10(6):981–91. https://doi.org/10.1007/s11548-015-1194-1

    Article  PubMed  Google Scholar 

  19. Ershad M, Rege R, Majewicz FA (2019) Automatic and near real-time stylistic behavior assessment in robotic surgery. Int J Comput Assist Radiol Surg 14(4):635–43. https://doi.org/10.1007/s11548-019-01920-6

    Article  PubMed  CAS  Google Scholar 

  20. Wang Z, Fey AM (2018) Deep learning with convolutional neural network for objective skill evaluation in robot-assisted surgery. Int J Comput Assist Radiol Surg 13(12):1959–70. https://doi.org/10.1007/s11548-018-1860-1

    Article  PubMed  Google Scholar 

  21. Brown JD, O’Brien CE, Leung SC, Dumon KR, Lee DI, Kuchenbecker KJ (2017) Using contact forces and robot arm accelerations to automatically rate surgeon skill at Peg transfer. IEEE Trans BioMed Eng 64(9):2263–75. https://doi.org/10.1109/tbme.2016.2634861

    Article  PubMed  Google Scholar 

  22. Ho TK (2002) A data complexity analysis of comparative advantages of decision forest constructors. Pattern Anal Appl 5(2):102–12. https://doi.org/10.1007/s100440200009

    Article  Google Scholar 

  23. Chen T, Guestrin C (2016) XGBoost: A scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining; San Francisco, California, USA: Association for Computing Machinery; 2016. pp 785–94

  24. Lundberg SM, Erion G, Chen H, DeGrave A, Prutkin JM, Nair B, Katz R, Himmelfarb J, Bansal N, Lee S-I (2020) From local explanations to global understanding with explainable AI for trees. Nat Mach Intell 2(1):56–67. https://doi.org/10.1038/s42256-019-0138-9

    Article  PubMed  PubMed Central  Google Scholar 

  25. Lundberg SM, Nair B, Vavilala MS, Horibe M, Eisses MJ, Adams T, Liston DE, Low DK-W, Newman S-F, Kim J, Lee S-I (2018) Explainable machine-learning predictions for the prevention of hypoxaemia during surgery. Nat Biomed Eng 2(10):749–60. https://doi.org/10.1038/s41551-018-0304-0

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Funding

Peking Union Medical College Graduate Student Innovation Fund (100232 01800402) and Capital’s Funds for Health Improvement and Research (2018–1-4041).

Author information

Authors and Affiliations

  1. School of Biomedical Science and Medical Engineering, Beihang University, Beijing, China

    Sutuke Yibulayimu, Yanzhen Liu, Zhibin Sun & Yu Wang

  2. Center of Body Contouring and Liposuction Center, Plastic Surgery Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China

    Yuneng Wang, Haiyue Jiang & Facheng Li

Authors
  1. Sutuke Yibulayimu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Yuneng Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Yanzhen Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhibin Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Yu Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Haiyue Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  7. Facheng Li
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Yu Wang.

Ethics declarations

Conflict of Interest

The authors, Sutuke Yibulayimu, Yuneng Wang, Yanzhen Liu, Zhibin Sun, Yu Wang, Haiyue Jiang, Facheng Li, declare that they have no conflicts of interest.

Ethical approval

Ethical approval was given by the Medical Ethics Committee of Plastic Surgery Hospital, Chinese Academy of Medical Sciences (2018–52).

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 122 KB)

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yibulayimu, S., Wang, Y., Liu, Y. et al. An explainable machine learning method for assessing surgical skill in liposuction surgery. Int J CARS 17, 2325–2336 (2022). https://doi.org/10.1007/s11548-022-02739-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-022-02739-4

Keywords