Abstract
Purpose
Automatic detection, classification and recording of operating room (OR) activities in real time during a surgical procedure requires a wide range of sensors to gather information on the activities of the surgeon and staff, the patient, and the OR equipment. The surgical instrument currently being used is an important parameter used to assess the instantaneous operating room status. An automated system was developed that detects unmodified surgical instruments in real time using a sensor-based table.
Methods
A multi-sensor operating room table was designed featuring a 2D camera, digital scale, and infrared camera. Software was developed to detect and record the sequence of changes on the table during a surgical procedure. The detection rates were evaluated under laboratory conditions by recording the observed instrument usage in 27 functional endoscopic sinus surgeries (FESS).
Results
The detection rate achieved using video-based detection alone was 84.9 %. The total detection rate achieved with the combined approach using both video- and weight-based information was 90.3 %.
Conclusion
A multi-sensor table-based automated instrument tracking system was developed that provides a foundation for the intra-operative detection of surgical instruments without modifying the instruments in the surgical tray. This system was tested and found to satisfy clinical FESS requirements with a reasonable accuracy. This system may also be useful to improve patient safety, for example to prevent instruments being left in the patient.
Similar content being viewed by others
References
Sutherland J, Van den Heuvel W (2006) Towards an intelligent hospital environment: adaptive workflow in the OR of the future. In: Proceedings of the 39th annual Hawaii international conference on system sciences, 2006, vol 5. HICSS’06, pp 100b–100b
Macario A, Vitez TS, Dunn B, McDonald T (1995) Where are the costs in perioperative care? Analysis of hospital costs and charges for inpatient surgical care. Anesthesiology 83(6):1138–1144
Archer T, Macario A (2006) The drive for operating room efficiency will increase quality of patient care. Curr Opin Anesthesiol 19(2):171–176
Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161
Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71
Cleary K, Kinsella A, Mun SK (2005) OR 2020 workshop report: operating room of the future. Int Congr Ser 1281:832–838
McKenzie C, Ibbotson J, Cao C (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invasive Ther Allied Technol 10(3):121–127
Kranzfelder M, Schneider A, Gillen S, Feussner H (2011) New technologies for information retrieval to achieve situational awareness and higher patient safety in the surgical operating room: the MRI institutional approach and review of the literature. Surg Endosc 25(3):696–705
Uecker DR, Wang YF, Lee C, Wang Y (1995) Laboratory investigation: automated instrument tracking in robotically assisted laparoscopic surgery. Comput Aided Surg 1(6):308–325
Speidel S, Delles M, Gutt C, Dillmann R (2006) Tracking of instruments in minimally invasive surgery for surgical skill analysis. In: Medical imaging and augmented reality. Springer, pp 148–155
Speidel S, Benzko J, Krappe S, Sudra G, Azad P, Peter B (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. Proc SPIE 7261:72610A–1
Voros S, Long J-A, Cinquin P (2006) Automatic localization of laparoscopic instruments for the visual servoing of an endoscopic camera holder. In: Medical image computing and computer-assisted intervention—MICCAI 2006. Springer, pp 535–542
Voros S, Long J-A, Cinquin P (2007) Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res 26(11–12):1173–1190
Sznitman R, Richa R, Taylor RH, Jedynak B, Hager GD (2013) Unified detection and tracking of instruments during retinal microsurgery. IEEE Trans Pattern Anal Mach Intell 35(5):1263–1273
Tonet O, Thoranaghatte RU, Megali G, Dario P (2007) Tracking endoscopic instruments without a localizer: a shape-analysis-based approach. Comput Aided Surg 12(1):35–42
Allen BF, Kasper F, Nataneli G, Dutson E, Faloutsos P (2011) Visual tracking of laparoscopic instruments in standard training environments. Stud Health Technol Inf 163:11–17
Bouarfa L, Akman O, Schneider A, Jonker PP, Dankelman J (2012) In-vivo real-time tracking of surgical instruments in endoscopic video. Minim Invasive Ther Allied Technol 21:129–134
Kranzfelder M, Schneider A, Blahusch G, Schaaf H, Feussner H (2009) Feasibility of opto-electronic surgical instrument identification. Minim Invasive Ther Allied Technol 18(5):253–258
Shepherd JP, Brickley MR, Jones ML (1994) Automatic identification of surgical and orthodontic instruments. Ann R Coll Surg Engl 76(2 Suppl):59–62
Rogers A, Jones E, Oleynikov D (2007) Radio frequency identification (RFID) applied to surgical sponges. Surg Endosc 21(7):1235–1237
Rivera N, Mountain R, Assumpcao L, Williams AA, Cooper AB, Lewis DL, Benson RC, Miragliotta JA, Marohn M, Taylor RH (2008) ASSIST-automated system for surgical instrument and sponge tracking. In: IEEE international conference on RFID, pp 297–302
Kranzfelder M, Zywitza D, Jell T, Schneider A, Gillen S, Friess H, Feussner H (2012) Real-time monitoring for detection of retained surgical sponges and team motion in the surgical operation room using radio-frequency-identification (RFID) technology: a preclinical evaluation. J Surg Res 175(2):191–198
Neumuth T, Meissner C (2012) Online recognition of surgical instruments by information fusion. Int J Comput Assist Radiol Surg 7(2):297–304
Meißner C, Neumuth T (2012) RFID-based surgical instrument detection using Hidden Markov models. Biomed Tech 57:1
Egan MT, Sandberg WS (2007) Auto identification technology and its impact on patient safety in the operating room of the future. Surg Innov 14(1):41–50
Lemke HU, Berliner L (2011) Systems design and management of the digital operating room. Int J Comput Assist Radiol Surg 6(1):144–158
Beeri Y, Einey D (2003) VISITS—Visual surgical instrument tracking system, 2003. [Online]. http://www.cs.huji.ac.il/course/2004/postPC/projects/fall2003/visits.html. Accessed 22 Apr 2014
Rattner DW, Park AE (2003) Advanced devices for the operating room of the future. Semin Laparosc Surg 10:85–90
Bobbit R, Connell J, Haas N, Otto C, Pankanti S, Payne J (2011) Visual item verification for fraud prevention in retail self-checkout. In: IEEE workshop on applications of computer vision (WACV), pp 585–590
Volosyak I, Radchenko O, Pape A, Martens C, She H, Wendland E, Gräser A (2003) Smart tray for the support of a wheelchair mounted manipulator. In: Proceedings of the international conference on economic, engineering and manufacturing systems. ICEEMS 2003
Volosyak I, Ivlev O, Graser A (2005) Rehabilitation robot FRIEND II-the general concept and current implementation. In: 9th International conference on rehabilitation robotics, 2005. ICORR 2005, pp 540–544
Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679–698
Pisinger D (2005) Where are the hard knapsack problems? Comput Oper Res 32(9):2271–2284
Neumuth T, Durstewitz N, Fischer M, Strauß G, Dietz A, Meixensberger J, Jannin P, Cleary K, Lemke H. U, Burgert O (2006) Structured recording of intraoperative surgical workflows. In: Medical imaging, pp 61450A–61450A
Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621
Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60
Liebmann P, Meixensberger J, Wiedemann P, Neumuth T (2013) The impact of missing sensor information on surgical workflow management. Int J Comput Assist Radiol Surg 8(5):867–875
Acknowledgments
The authors would like to thank the following people who influenced this work with their kind support and dedication: From the Acqua Klinik and International Reference and Development Center (IRDC) in Leipzig (Germany): Gero Strauß, Iris Gollnick, Susanne Modemann, Wassem An-Nayef, Mengiste Frehiwot, Anja Rothe, Juliane Apitzsch, Tina Rönnspiess, Jacqueline Schlesier, and Susanne Schiele. From the Innovation Center Computer Assisted Surgery at the University of Leipzig (Germany): Stefan Franke, Gerald Sommer, Michael Unger, Nadine Pohle, and Alexandra Lebelt. From the Central Sterilization Department at the University Medical Center of Leipzig (Germany): Kerstin Schröter. ICCAS is funded by the German Federal Ministry of Education and Research (BMBF), the Saxon Ministry of Science and Fine Arts (SMWK) in the Unternehmen Region with Grant No. 03Z1LN12, the European Regional Development Fund (ERDF) and the state of Saxony within the framework of measures to support the technology sector.
Ethical standards
The manuscript does not contain clinical studies or patient data.
Conflict of interest
The authors declare that they have no conflict of interest.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Glaser, B., Dänzer, S. & Neumuth, T. Intra-operative surgical instrument usage detection on a multi-sensor table. Int J CARS 10, 351–362 (2015). https://doi.org/10.1007/s11548-014-1066-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-014-1066-0