Skip to main content

Advertisement

SpringerLink
Log in

Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

Computer assistance is increasingly common in surgery. However, the amount of information is bound to overload processing abilities of surgeons. We propose methods to recognize the current phase of a surgery for context-aware information filtering. The purpose is to select the most suitable subset of information for surgical situations which require special assistance.

Methods

We combine formal knowledge, represented by an ontology, and experience-based knowledge, represented by training samples, to recognize phases. For this purpose, we have developed two different methods. Firstly, we use formal knowledge about possible phase transitions to create a composition of random forests. Secondly, we propose a method based on cultural optimization to infer formal rules from experience to recognize phases.

Results

The proposed methods are compared with a purely formal knowledge-based approach using rules and a purely experience-based one using regular random forests. The comparative evaluation on laparoscopic pancreas resections and adrenalectomies employs a consistent set of quality criteria on clean and noisy input. The rule-based approaches proved best with noisefree data. The random forest-based ones were more robust in the presence of noise.

Conclusion

Formal and experience-based knowledge can be successfully combined for robust phase recognition.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Skagestad P (1993) Thinking with machines: intelligence augmentation, evolutionary epistemology, and semiotic. J Soc Evol Syst 16(2):157–180

    Article  Google Scholar 

  2. Woods DD, Patterson ES, Roth EM (2002) Can we ever escape from data overload? A cognitive systems diagnosis. Cogn Technol Work 4(1):22–36

    Article  Google Scholar 

  3. Dixon B, Daly MJ, Chan H, Vescan AD, Witterick IJ, Irish JC (2013) Surgeons blinded by enhanced navigation: the effect of augmented reality on attention. Surg Endosc 27(2):454–461

    Article  PubMed  Google Scholar 

  4. Joyce JP, Lapinsky GW (1983) A history and overview of the safety parameter display system concept. IEEE Trans Nucl Sci 30(1):744–749

    Article  Google Scholar 

  5. Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27(5):1681–1688

    Article  PubMed  Google Scholar 

  6. Linte CA, Davenport KP, Cleary K, Peters C, Vosburgh KG, Navab N, Edwards PE, Jannin P, Peters TM, Holmes DR, Robb RA (2013) On mixed reality environments for minimally invasive therapy guidance: systems architecture, successes and challenges in their implementation from laboratory to clinic. Comp Med Imaging Graph 37(2):83–97

    Article  Google Scholar 

  7. Lalys F, Jannin P (2014) Surgical process modelling: a review. Int J Comput Assist Radiol Surg 9(3):495–511. doi:10.1007/s11548-013-0940-5

  8. Blum T, Feussner H, Navab N (2010) Modeling and segmentation of surgical workflow from laparoscopic video. In: Jiang T, Navab N, Pluim JPW, Viergever MA (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2010, vol 6363. Springer, Berlin, pp 400–407

  9. Suzuki T, Sakurai Y, Yoshimitsu K, Nambu K, Muragaki Y, Iseki H (2010) Intraoperative multichannel audio–visual information recording and automatic surgical phase and incident detection. In: IEEE EMBS, pp 1190–1193

  10. Blum T, Padoy N, Feussner H, Navab N (2008) Workflow mining for visualization and analysis of surgeries. Int J Comput Assist Radiol Surg 3(5):379–386

    Article  Google Scholar 

  11. Bouarfa L, Jonker PP, Dankelman J (2010) Discovery of high-level tasks in the operating room. J Biomed Inform 44(3):455–462

    Article  PubMed  Google Scholar 

  12. Ahmadi A, Sielhorst T, Stauder R, Horn M, Feussner H, Navab N (2007) Recovery of surgical workflow without explicit models. In: Proceedings of the MICCAI, pp 420–428

  13. Stauder R, Okur A, Peter L, Schneider A, Kranzfelder M, Feussner H, Navab N (2014) Random forests for phase detection in surgical workflow analysis. In: Proceedings of the IPCAI

  14. Reiley CE, Lin HC, Varadarajan B, Vagvolgyi B, Khudanpur S, Yuh DD, Hager GD (2008) Automatic recognition of surgical motions using statistical modeling for capturing variability. Stud Health Technol Inform 132:396–401

  15. Lalys F, Riffaud L, Morandi X, Jannin P (2011) Surgical phases detection from microscope videos by combining SVM and HMM medical computer vision. Recognition techniques and applications in medical imaging

  16. Padoy N, Blum T, Ahmadi SA, Feussner H, Berger MO, Navab N (2010) Statistical modeling and recognition of surgical workflow. Med Image Anal 16(3):632–641

    Article  PubMed  Google Scholar 

  17. Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71

    Article  PubMed  PubMed Central  Google Scholar 

  18. Burgert O, Neumuth T, Lempp F, Mudunuri R, Meixensberger J, Strauss G, Dietz A, Jannin P, Lemke HU (2006) Linking top-level ontologies and surgical workflows. Int J Comput Assist Radiol Surg 1(1):437–438

    Google Scholar 

  19. Jannin P, Morandi X (2007) Surgical models for computer-assisted neurosurgery. Neuroimage 37(3):783–791

    Article  CAS  PubMed  Google Scholar 

  20. Neumuth T, Kaschek B, Neumuth D, Ceschia M, Meixensberger J, Strauss G, Burgert O (2010) An observation support system with an adaptive ontology-driven user interface for the modeling of complex behaviors during surgical interventions. Behav Res Methods 42(4):1049–1058

    Article  CAS  PubMed  Google Scholar 

  21. Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161

    Article  PubMed  Google Scholar 

  22. Lalys F, Bouget B, Riffaud R, Jannin P (2013) Automatic knowledge-based recognition of low-level tasks in ophthalmological procedures. Int J CARS 8:39–49

    Article  Google Scholar 

  23. Katic D, Julliard C, Wekerle AL, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J CARS 10:1427–1434

    Article  Google Scholar 

  24. Katic D, Wekerle AL, Grtner F, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S (2014) Knowledge-driven formalization of laparoscopic surgeries for rule-based intraoperative context-aware assistance. In: IPCAI

  25. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  26. Katic D, Wekerle AL, Gaertner F, Kenngott H, Müller-Stich BP, Dillmann E, Speidel S (2013) Ontology-based prediction of surgical events in laparoscopic surgery. In: Proceedings of the SPIE

  27. Forestier G, Riffaud L, Jannin P (2015) Automatic phase prediction from low-level surgical activities. Int J Comput Assist Radiol Surg 20(6):833–884

    Article  Google Scholar 

  28. Horrocks I, Patel-Schneider PF, Boley H, Tabet S, Grosof B, Dean M (2004) SWRL: a semantic web rule language combining OWL and RuleML. W3C Member Submission. https://www.w3.org/Submission/SWRL/. Accessed 25 Mar 2016

  29. Ma H, Wang y (2009) Cultural algorithm based on particle swarm optimization for function optimization. In: Fifth international conference on natural computation

  30. Kennedy J, Eberhart R (1995) Particle swarm optimization. In: IEEE international conference on NN

  31. Kobti Z, Reynolds R, Kohler T (2004) Agent-based modeling of cultural change in swarm using cultural algorithms. In: SWARMFEST

  32. Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inform Assoc 16(1):72–80

  33. Speidel S, Benzko J, Sudra G, Azad P, Müller-Stich BP, Gutt , Dillmann R (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. In: SPIE medical imaging

  34. Yang Y, Calmet J (2005) OntoBayes: an ontology-driven uncertainty model. In: International conference on intelligent agents, web technologies and internet commerce

Download references

Acknowledgments

The present research was supported by the “SFB TRR 125” funded by the DFG the ESF of Baden-Wuerttemberg.

Author information

Authors and Affiliations

  1. Institute for Anthropomatics and Robotics (IAR), Karlsruhe Institute of Technology (KIT), Karlsruhe, Germany

    Darko Katić, Jürgen Schuck, Rüdiger Dillmann & Stefanie Speidel

  2. Department of General, Abdominal and Transplantation Surgery, University of Heidelberg, Heidelberg, Germany

    Anna-Laura Wekerle, Hannes Kenngott & Beat Peter Müller-Stich

Authors
  1. Darko Katić
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jürgen Schuck
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Anna-Laura Wekerle
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hannes Kenngott
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Beat Peter Müller-Stich
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Rüdiger Dillmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Stefanie Speidel
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Darko Katić.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical standard

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki Declaration and its later amendments or comparable ethical standards. For this type of study formal consent was not required since only secondary data, namely performed annotations without patient identifiers, were used.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Katić, D., Schuck, J., Wekerle, AL. et al. Bridging the gap between formal and experience-based knowledge for context-aware laparoscopy. Int J CARS 11, 881–888 (2016). https://doi.org/10.1007/s11548-016-1379-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-016-1379-2

Keywords

Search

Navigation