Cautery tool state detection using deep learning on intraoperative surgery videos

L. March; J. R. Rodgers; R. Hisey; A. Jamzad; A. M. L. Santilli; D. McKay; J. F. Rudan; M. Kaufmann; K. Y. M. Ren; G. Fichtinger; P. Mousavi

doi:10.1117/12.2654234

3 April 2023 Cautery tool state detection using deep learning on intraoperative surgery videos

L. March, J. R. Rodgers, R. Hisey, A. Jamzad, A. M. L. Santilli, D. McKay, J. F. Rudan, M. Kaufmann, K. Y. M. Ren, G. Fichtinger, P. Mousavi

Author Affiliations +

Proceedings Volume 12466, Medical Imaging 2023: Image-Guided Procedures, Robotic Interventions, and Modeling; 124660D (2023) https://doi.org/10.1117/12.2654234
Event: SPIE Medical Imaging, 2023, San Diego, California, United States

Abstract

Treatment for Basal Cell Carcinoma (BCC) includes an excisional surgery to remove cancerous tissues, using a cautery tool to make burns along a defined resection margin around the tumor. Margin evaluation occurs post-surgically, requiring repeat surgery if positive margins are detected. Rapid Evaporative Ionization Mass Spectrometry (REIMS) can help distinguish healthy and cancerous tissue but does not provide spatial information about the cautery tool location where the spectra are acquired. We propose using intraoperative surgical video recordings and deep learning to provide surgeons with guidance to locate sites of potential positive margins. Frames from 14 intraoperative videos of BCC surgery were extracted and used to train a sequence of networks. The first network extracts frames showing surgery in-progress, then, an object detection network localizes the cautery tool and resection margin. Finally, our burn prediction model leverages the effectiveness of both a Long Short-Term Memory (LSTM) network and a Receiver Operating Characteristic (ROC) curve to accurately predict when the surgeon is cutting. The cut identifications will be used in the future for synchronization with iKnife data to provide localizations when cuts are predicted. The model was trained with four-fold cross-validation on a patient-wise split between training, validation, and testing sets. Average recall over the four folds of testing for the LSTM and ROC were 0.80 and 0.73, respectively. The video-based approach is simple yet effective at identifying tool-to-skin contact instances and may help guide surgeons, enabling them to deliver precise treatments in combination with iKnife data.

Conference Presentation

Citation Download Citation

L. March, J. R. Rodgers, R. Hisey, A. Jamzad, A. M. L. Santilli, D. McKay, J. F. Rudan, M. Kaufmann, K. Y. M. Ren, G. Fichtinger, and P. Mousavi "Cautery tool state detection using deep learning on intraoperative surgery videos", Proc. SPIE 12466, Medical Imaging 2023: Image-Guided Procedures, Robotic Interventions, and Modeling, 124660D (3 April 2023); https://doi.org/10.1117/12.2654234

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available