Skip to main content

Advertisement

Log in

Intra-operative surgical instrument usage detection on a multi-sensor table

  • Original Article
  • Published:
International Journal of Computer Assisted Radiology and Surgery Aims and scope Submit manuscript

Abstract

Purpose

 Automatic detection, classification and recording of operating room (OR) activities in real time during a surgical procedure requires a wide range of sensors to gather information on the activities of the surgeon and staff, the patient, and the OR equipment. The surgical instrument currently being used is an important parameter used to assess the instantaneous operating room status. An automated system was developed that detects unmodified surgical instruments in real time using a sensor-based table.

Methods

 A multi-sensor operating room table was designed featuring a 2D camera, digital scale, and infrared camera. Software was developed to detect and record the sequence of changes on the table during a surgical procedure. The detection rates were evaluated under laboratory conditions by recording the observed instrument usage in 27 functional endoscopic sinus surgeries (FESS).

Results

 The detection rate achieved using video-based detection alone was 84.9 %. The total detection rate achieved with the combined approach using both video- and weight-based information was 90.3 %.

Conclusion

 A multi-sensor table-based automated instrument tracking system was developed that provides a foundation for the intra-operative detection of surgical instruments without modifying the instruments in the surgical tray. This system was tested and found to satisfy clinical FESS requirements with a reasonable accuracy. This system may also be useful to improve patient safety, for example to prevent instruments being left in the patient.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Sutherland J, Van den Heuvel W (2006) Towards an intelligent hospital environment: adaptive workflow in the OR of the future. In: Proceedings of the 39th annual Hawaii international conference on system sciences, 2006, vol 5. HICSS’06, pp 100b–100b

  2. Macario A, Vitez TS, Dunn B, McDonald T (1995) Where are the costs in perioperative care? Analysis of hospital costs and charges for inpatient surgical care. Anesthesiology 83(6):1138–1144

    Article  CAS  PubMed  Google Scholar 

  3. Archer T, Macario A (2006) The drive for operating room efficiency will increase quality of patient care. Curr Opin Anesthesiol 19(2):171–176

    Article  Google Scholar 

  4. Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161

    Article  PubMed  Google Scholar 

  5. Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71

    Article  PubMed Central  PubMed  Google Scholar 

  6. Cleary K, Kinsella A, Mun SK (2005) OR 2020 workshop report: operating room of the future. Int Congr Ser 1281:832–838

    Article  Google Scholar 

  7. McKenzie C, Ibbotson J, Cao C (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invasive Ther Allied Technol 10(3):121–127

    Article  Google Scholar 

  8. Kranzfelder M, Schneider A, Gillen S, Feussner H (2011) New technologies for information retrieval to achieve situational awareness and higher patient safety in the surgical operating room: the MRI institutional approach and review of the literature. Surg Endosc 25(3):696–705

    Article  PubMed  Google Scholar 

  9. Uecker DR, Wang YF, Lee C, Wang Y (1995) Laboratory investigation: automated instrument tracking in robotically assisted laparoscopic surgery. Comput Aided Surg 1(6):308–325

    Article  CAS  Google Scholar 

  10. Speidel S, Delles M, Gutt C, Dillmann R (2006) Tracking of instruments in minimally invasive surgery for surgical skill analysis. In: Medical imaging and augmented reality. Springer, pp 148–155

  11. Speidel S, Benzko J, Krappe S, Sudra G, Azad P, Peter B (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. Proc SPIE 7261:72610A–1

  12. Voros S, Long J-A, Cinquin P (2006) Automatic localization of laparoscopic instruments for the visual servoing of an endoscopic camera holder. In: Medical image computing and computer-assisted intervention—MICCAI 2006. Springer, pp 535–542

  13. Voros S, Long J-A, Cinquin P (2007) Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res 26(11–12):1173–1190

    Article  Google Scholar 

  14. Sznitman R, Richa R, Taylor RH, Jedynak B, Hager GD (2013) Unified detection and tracking of instruments during retinal microsurgery. IEEE Trans Pattern Anal Mach Intell 35(5):1263–1273

    Article  PubMed  Google Scholar 

  15. Tonet O, Thoranaghatte RU, Megali G, Dario P (2007) Tracking endoscopic instruments without a localizer: a shape-analysis-based approach. Comput Aided Surg 12(1):35–42

    Article  PubMed  Google Scholar 

  16. Allen BF, Kasper F, Nataneli G, Dutson E, Faloutsos P (2011) Visual tracking of laparoscopic instruments in standard training environments. Stud Health Technol Inf 163:11–17

    Google Scholar 

  17. Bouarfa L, Akman O, Schneider A, Jonker PP, Dankelman J (2012) In-vivo real-time tracking of surgical instruments in endoscopic video. Minim Invasive Ther Allied Technol 21:129–134

    Article  PubMed  Google Scholar 

  18. Kranzfelder M, Schneider A, Blahusch G, Schaaf H, Feussner H (2009) Feasibility of opto-electronic surgical instrument identification. Minim Invasive Ther Allied Technol 18(5):253–258

    Article  PubMed  Google Scholar 

  19. Shepherd JP, Brickley MR, Jones ML (1994) Automatic identification of surgical and orthodontic instruments. Ann R Coll Surg Engl 76(2 Suppl):59–62

    CAS  PubMed  Google Scholar 

  20. Rogers A, Jones E, Oleynikov D (2007) Radio frequency identification (RFID) applied to surgical sponges. Surg Endosc 21(7):1235–1237

    Article  CAS  PubMed  Google Scholar 

  21. Rivera N, Mountain R, Assumpcao L, Williams AA, Cooper AB, Lewis DL, Benson RC, Miragliotta JA, Marohn M, Taylor RH (2008) ASSIST-automated system for surgical instrument and sponge tracking. In: IEEE international conference on RFID, pp 297–302

  22. Kranzfelder M, Zywitza D, Jell T, Schneider A, Gillen S, Friess H, Feussner H (2012) Real-time monitoring for detection of retained surgical sponges and team motion in the surgical operation room using radio-frequency-identification (RFID) technology: a preclinical evaluation. J Surg Res 175(2):191–198

    Article  PubMed  Google Scholar 

  23. Neumuth T, Meissner C (2012) Online recognition of surgical instruments by information fusion. Int J Comput Assist Radiol Surg 7(2):297–304

    Article  PubMed  Google Scholar 

  24. Meißner C, Neumuth T (2012) RFID-based surgical instrument detection using Hidden Markov models. Biomed Tech 57:1

  25. Egan MT, Sandberg WS (2007) Auto identification technology and its impact on patient safety in the operating room of the future. Surg Innov 14(1):41–50

    Article  PubMed  Google Scholar 

  26. Lemke HU, Berliner L (2011) Systems design and management of the digital operating room. Int J Comput Assist Radiol Surg 6(1):144–158

    Google Scholar 

  27. Beeri Y, Einey D (2003) VISITS—Visual surgical instrument tracking system, 2003. [Online]. http://www.cs.huji.ac.il/course/2004/postPC/projects/fall2003/visits.html. Accessed 22 Apr 2014

  28. Rattner DW, Park AE (2003) Advanced devices for the operating room of the future. Semin Laparosc Surg 10:85–90

    PubMed  Google Scholar 

  29. Bobbit R, Connell J, Haas N, Otto C, Pankanti S, Payne J (2011) Visual item verification for fraud prevention in retail self-checkout. In: IEEE workshop on applications of computer vision (WACV), pp 585–590

  30. Volosyak I, Radchenko O, Pape A, Martens C, She H, Wendland E, Gräser A (2003) Smart tray for the support of a wheelchair mounted manipulator. In: Proceedings of the international conference on economic, engineering and manufacturing systems. ICEEMS 2003

  31. Volosyak I, Ivlev O, Graser A (2005) Rehabilitation robot FRIEND II-the general concept and current implementation. In: 9th International conference on rehabilitation robotics, 2005. ICORR 2005, pp 540–544

  32. Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679–698

    Article  Google Scholar 

  33. Pisinger D (2005) Where are the hard knapsack problems? Comput Oper Res 32(9):2271–2284

    Article  Google Scholar 

  34. Neumuth T, Durstewitz N, Fischer M, Strauß G, Dietz A, Meixensberger J, Jannin P, Cleary K, Lemke H. U, Burgert O (2006) Structured recording of intraoperative surgical workflows. In: Medical imaging, pp 61450A–61450A

  35. Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621

    Article  Google Scholar 

  36. Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60

    Article  Google Scholar 

  37. Liebmann P, Meixensberger J, Wiedemann P, Neumuth T (2013) The impact of missing sensor information on surgical workflow management. Int J Comput Assist Radiol Surg 8(5):867–875

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors would like to thank the following people who influenced this work with their kind support and dedication: From the Acqua Klinik and International Reference and Development Center (IRDC) in Leipzig (Germany): Gero Strauß, Iris Gollnick, Susanne Modemann, Wassem An-Nayef, Mengiste Frehiwot, Anja Rothe, Juliane Apitzsch, Tina Rönnspiess, Jacqueline Schlesier, and Susanne Schiele. From the Innovation Center Computer Assisted Surgery at the University of Leipzig (Germany): Stefan Franke, Gerald Sommer, Michael Unger, Nadine Pohle, and Alexandra Lebelt. From the Central Sterilization Department at the University Medical Center of Leipzig (Germany): Kerstin Schröter. ICCAS is funded by the German Federal Ministry of Education and Research (BMBF), the Saxon Ministry of Science and Fine Arts (SMWK) in the Unternehmen Region with Grant No. 03Z1LN12, the European Regional Development Fund (ERDF) and the state of Saxony within the framework of measures to support the technology sector.

Ethical standards

The manuscript does not contain clinical studies or patient data.

Conflict of interest

The authors declare that they have no conflict of interest.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Bernhard Glaser.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Glaser, B., Dänzer, S. & Neumuth, T. Intra-operative surgical instrument usage detection on a multi-sensor table. Int J CARS 10, 351–362 (2015). https://doi.org/10.1007/s11548-014-1066-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11548-014-1066-0

Keywords

Navigation