Abstract
Near infrared fluorescence optical imaging (NIR-FOI) is a relatively new imaging modality to diagnose arthritis in the hands. The acquired data has two spatial dimensions and one temporal dimension, which visualizes the time dependent distribution of an administered color agent. In accordance with previous work, we hypothesize that the distribution process allows a joint-wise classification into inflammatory affected and unaffected.
In this work, we present the first approach to objectively classify hand joint NIR-FOI image stacks by designing, training, and testing a neural network. Previously presented model architectures for spatio-temporal classification do not yield satisfying results when trained on NIR-FOI data. A recall value of 0.812 of the over- and a recall value of 0.652 of the underrepresented class is achieved, the model’s robustness tested against small variations and its attention visualized in activation maps.
Even though these results leave room for further improvement, they also indicate, that the model architecture can capture the latent features of the data. We are confident, that more available data will lead to a robust classification model and can support medical doctors in using NIR-FOI as a diagnostic tool for PsA.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ballas, N., Yao, L., Pal, C., Courville, A.: Delving deeper into convolutional networks for learning video representations. arxiv.org/abs/1511.06432 (2015)
D’Angiolella, L.S., et al.: Cost and cost effectiveness of treatments for psoriatic arthritis: a systematic literature review. PharmacoEconomics 36(5), 567–589 (2018). https://doi.org/10.1007/s40273-018-0618-5
Gladman, D.D.: Psoriatic arthritis: epidemiology, clinical features, course, and outcome. Ann. Rheum. Dis. 64(suppl 2), ii14–ii17 (2005). https://doi.org/10.1136/ard.2004.032482
Halder, R., Chatterjee, R.: CNN-BiLSTM model for violence detection in smart surveillance. SN Comput. Sci. 1(4) (2020). https://doi.org/10.1007/s42979-020-00207-x
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997). https://doi.org/10.1162/neco.1997.9.8.1735
Lea, C., Flynn, M.D., Vidal, R., Reiter, A., Hager, G.D.: Temporal convolutional networks for action segmentation and detection. arxiv.org/abs/1611.05267 (2016)
Mao, F., Wu, X., Xue, H., Zhang, R.: Hierarchical video frame sequence representation with deep convolutional graph network. In: ECCV 2018 Workshops, pp. 262–270 (2019). https://doi.org/10.1007/978-3-030-11018-5_24
Paul, S.: Video classification with a CNN-RNN architecture (2021). https://keras.io/examples/vision/video_classification/. Accessed 25th Apr 2023
Werner, S.G., et al.: Indocyanine green-enhanced fluorescence optical imaging in patients with early and very early arthritis: a comparative study with magnetic resonance imaging. Arthritis Rheum. 65(12), 3036–3044 (2013). https://doi.org/10.1002/art.38175
Acknowledgment
This project has received funding from the Innovative Medicines Initiative 2 Joint Undertaking (JU) under grant agreement No 101 007 757 (Hippocrates). The JU receives support from the European Union’s Horizon 2020 research and innovation program and EFPIA.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Zerweck, L., Wesarg, S., Kohlhammer, J., Köhm, M. (2023). Automated Hand Joint Classification of Psoriatic Arthritis Patients Using Routinely Acquired Near Infrared Fluorescence Optical Imaging. In: Wesarg, S., et al. Clinical Image-Based Procedures, Fairness of AI in Medical Imaging, and Ethical and Philosophical Issues in Medical Imaging. CLIP EPIMI FAIMI 2023 2023 2023. Lecture Notes in Computer Science, vol 14242. Springer, Cham. https://doi.org/10.1007/978-3-031-45249-9_1
Download citation
DOI: https://doi.org/10.1007/978-3-031-45249-9_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45248-2
Online ISBN: 978-3-031-45249-9
eBook Packages: Computer ScienceComputer Science (R0)
