Automatic labeling of vertebrae in long-length intraoperative imaging with a multi-view, region-based CNN

Y. Huang; C. K. Jones; X. Zhang; A. Johnston; N. Aygun; T. F. Witham; P. A. Helm; J. H. Siewerdsen; A. Uneri

doi:10.1117/12.2611912

4 April 2022 Automatic labeling of vertebrae in long-length intraoperative imaging with a multi-view, region-based CNN

Y. Huang, C. K. Jones, X. Zhang, A. Johnston, N. Aygun, T. F. Witham, P. A. Helm, J. H. Siewerdsen, A. Uneri

Proceedings Volume 12034, Medical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling; 120340U (2022) https://doi.org/10.1117/12.2611912
Event: SPIE Medical Imaging, 2022, San Diego, California, United States

Abstract

Purpose: A recent imaging method (viz., Long-Film) for capturing long-length images of the spine was enabled on the Oarm™ system. Proposed work uses a custom, multi-perspective, region-based convolutional neural network (R-CNN) for labeling vertebrae in Long-Film images and evaluates approaches for incorporating long contextual information to take advantage of the extended field-of-view and improve the labeling accuracy. Methods: Evaluated methods for incorporating contextual information include: (1) a recurrent network module with long short-term memory (LSTM) added after R-CNN classification; and (2) a post-processing, sequence-sorting step based on the label confidence scores. The models were trained and validated on 11,805 Long-Film images simulated from projections of 370 CT images and tested on 50 Long-Film images of 14 cadaveric specimens. Results: The multi-perspective R-CNN with LSTM module achieved 91.7% vertebrae level identification rate, compared to 72.4% when used without LSTM, thus demonstrating the improvement of incorporating contextual information. While sequence sorting achieved 89.4% in labeling accuracy, it failed to handle errors during detection and did not provide additional improvements when applied following the LSTM module. Conclusions: The proposed LSTM module significantly improved the labeling accuracy upon the base model through effective contextual information incorporation and training in an end-to-end fashion. Compared to sequence sorting, it showed more flexibility towards false positives and false negatives in vertebrae detection. The proposed model offers the potential to provide a valuable check for target localization and forms the basis for automatic measurement of spinal curvature changes in interventional settings.

Conference Presentation

Citation Download Citation

Y. Huang, C. K. Jones, X. Zhang, A. Johnston, N. Aygun, T. F. Witham, P. A. Helm, J. H. Siewerdsen, and A. Uneri "Automatic labeling of vertebrae in long-length intraoperative imaging with a multi-view, region-based CNN", Proc. SPIE 12034, Medical Imaging 2022: Image-Guided Procedures, Robotic Interventions, and Modeling, 120340U (4 April 2022); https://doi.org/10.1117/12.2611912

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available