OsteoGA: An Explainable AI Framework for Knee Osteoarthritis Severity Assessment
Authors: 
Hieu Phan Trung, Su Nguyen Thiet, Tuan Nguyen Trung, Loc Le Tan, Triet Tran Minh, Tho Quan ThanhAuthors Info & Claims
Hieu Phan Trung, Su Nguyen Thiet, Tuan Nguyen Trung, Loc Le Tan, Triet Tran Minh, Tho Quan ThanhAuthors Info & ClaimsPages 639 - 646
Abstract
Knee osteoarthritis is among the most common joint disorders. Recent studies have investigated the application of Artificial Intelligence (AI) in automated diagnosis using knee joint X-ray images. However, these studies have primarily focused on diagnosing the severity of osteoarthritis without providing explanations for the underlying reasons that led to those results. In this paper, we present OsteoGA, an AI framework that focuses on the interpretability of the model in order to assist in the diagnosis of knee osteoarthritis. OsteoGA introduces a novel generative adversarial autoencoder model, called GAE, to reconstruct an assumed healthy knee joint image from the original ones. The reconstruction process in OsteoGA combines image reconstruction, data imputation, and adversarial learning to generate high-quality images that are consistent and coherent with the patient’s image. Apart from effectively diagnosing the severity of knee osteoarthritis, the OsteoGA framework also produces an anomaly map. This map highlights valuable information about the abnormal regions in X-ray images, offering additional insights to medical experts during the diagnosis process.
References
[1]
Omar Ali, Wiem Abdelbaki, Anup Shrestha, Ersin Elbasi, Mohammad Abdallah Ali Alryalat, and Yogesh K Dwivedi. 2023. A systematic literature review of artificial intelligence in the healthcare sector: Benefits, challenges, methodologies, and functionalities. Journal of Innovation & Knowledge 8, 1 (2023), 100333.
[2]
Esam Alshareef, Fawzi Omar, Yosra Lamami, Mohamed Milad, Mohamed Eswani, Sedigh Bashir, Salah Bshina, Anas Jakdoum, Asharaf Abourqeeqah, Mohamed Elbasir, and Ellafi.A. Elbahrit. 2022. Knee osteoarthritis severity grading using vision transformer. Journal of Intelligent & Fuzzy Systems 43 (08 2022), 1–11. https://doi.org/10.3233/JIFS-220516
[3]
Ulfah Aprilliani and Zuherman Rustam. 2018. Osteoarthritis Disease Prediction Based on Random Forest. In 2018 International Conference on Advanced Computer Science and Information Systems (ICACSIS). 237–240. https://doi.org/10.1109/ICACSIS.2018.8618166
[4]
Martin Arjovsky, Soumith Chintala, and Léon Bottou. 2017. Wasserstein Generative Adversarial Networks. In Proceedings of the 34th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 70), Doina Precup and Yee Whye Teh (Eds.). PMLR, 214–223. https://proceedings.mlr.press/v70/arjovsky17a.html
[5]
Haoyu Chen, Jinjin Gu, Yihao Liu, Salma Magid, Chao Dong, Qiong Wang, Hanspeter Pfister, and Lei Zhu. 2023. Masked Image Training for Generalizable Deep Image Denoising. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 1692–1703. https://doi.org/10.1109/CVPR52729.2023.00169
[6]
Aiyong Cui, Huizi Li, Dawei Wang, Junlong Zhong, Yufeng Chen, and Huading Lu. 2020. Global, regional prevalence, incidence and risk factors of knee osteoarthritis in population-based studies. EClinicalMedicine 29-30 (12 2020), 100587. https://doi.org/10.1016/j.eclinm.2020.100587
[7]
Keyan Ding, Kede Ma, Shiqi Wang, and Eero P. Simoncelli. 2022. Image Quality Assessment: Unifying Structure and Texture Similarity. IEEE Transactions on Pattern Analysis and Machine Intelligence 44, 5 (2022), 2567–2581. https://doi.org/10.1109/TPAMI.2020.3045810
[8]
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. CoRR abs/2010.11929 (2020). arXiv:2010.11929https://arxiv.org/abs/2010.11929
[9]
Mohammad Hossein Givkashi, Mahshid Hadipour, Arezoo PariZanganeh, Zahra Nabizadeh, Nader Karimi, and Shadrokh Samavi. 2022. Image Inpainting Using AutoEncoder and Guided Selection of Predicted Pixels. In 2022 30th International Conference on Electrical Engineering (ICEE). IEEE, 700–704. https://doi.org/10.1109/ICEE55646.2022.9827427
[10]
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Advances in neural information processing systems. 2672–2680.
[11]
Hanxue Gu, Keyu Li, Roy J Colglazier, Jichen Yang, Michael Lebhar, Jonathan O’Donnell, William A Jiranek, Richard C Mather, Rob J French, Nicholas Said, 2022. Knee arthritis severity measurement using deep learning: a publicly available algorithm with a multi-institutional validation showing radiologist-level performance. arXiv preprint arXiv:2203.08914 (2022).
[12]
Rohit Gupta, Anurag Sharma, and Anupam Kumar. 2020. Super-Resolution using GANs for Medical Imaging. Procedia Computer Science 173 (2020), 28–35. https://doi.org/10.1016/j.procs.2020.06.005 International Conference on Smart Sustainable Intelligent Computing and Applications under ICITETM2020.
[13]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770–778. https://doi.org/10.1109/CVPR.2016.90
[14]
Lan T Ho-Pham, Thai Q Lai, Linh D Mai, Minh C Doan, Hoa N Pham, and Tuan V Nguyen. 2014. Prevalence of radiographic osteoarthritis of the knee and its relationship to self-reported pain. PloS one 9, 4 (2014), e94563.
[15]
Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2017. Densely Connected Convolutional Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 2261–2269. https://doi.org/10.1109/CVPR.2017.243
[16]
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. 2017. Image-to-image translation with conditional adversarial networks. In the IEEE conference on computer vision and pattern recognition (CVPR). 1125–1134.
[17]
Rohit Kumar Jain, Prasen Kumar Sharma, Sibaji Gaj, Arijit Sur, and Palash Ghosh. 2023. Knee osteoarthritis severity prediction using an attentive multi-scale deep convolutional neural network. Multimedia Tools and Applications (2023), 1–18.
[18]
Yifan Jiang, Shiyu Chang, and Zhangyang Wang. 2021. TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up. Advances in Neural Information Processing Systems 34 (2021).
[19]
Tero Karras, Samuli Laine, and Timo Aila. 2019. A Style-Based Generator Architecture for Generative Adversarial Networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).
[20]
Jonas H Kellgren and J. S. Lawrence. 1957. Radiological assessment of osteo-arthrosis. Annals of the rheumatic diseases 16, 4 (1957), 494.
[21]
Michelle J Lespasio, Nicolas S Piuzzi, M Elaine Husni, George F Muschler, AJ Guarino, and Michael A Mont. 2017. Knee osteoarthritis: a primer. The Permanente Journal 21 (2017).
[22]
Zhuang Liu, Hanzi Mao, Chao-Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. 2022. A ConvNet for the 2020s. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 11966–11976. https://doi.org/10.1109/CVPR52688.2022.01167
[23]
Iaroslav Melekhov, Juho Kannala, and Esa Rahtu. 2016. Siamese network features for image matching. In 2016 23rd international conference on pattern recognition (ICPR). IEEE, 378–383.
[24]
Huy Hoang Nguyen, Simo Saarakkala, Matthew B. Blaschko, and Aleksei Tiulpin. 2020. Semixup: In- and Out-of-Manifold Regularization for Deep Semi-Supervised Knee Osteoarthritis Severity Grading From Plain Radiographs. IEEE Transactions on Medical Imaging 39, 12 (2020), 4346–4356.
[25]
Alec Radford, Luke Metz, and Soumith Chintala. 2015. Unsupervised representation learning with deep convolutional generative adversarial networks. arXiv preprint arXiv:1511.06434 (2015).
[26]
Ilija Radosavovic, Raj Prateek Kosaraju, Ross Girshick, Kaiming He, and Piotr Dollár. 2020. Designing Network Design Spaces. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 10425–10433. https://doi.org/10.1109/CVPR42600.2020.01044
[27]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015, Nassir Navab, Joachim Hornegger, William M. Wells, and Alejandro F. Frangi (Eds.). Springer International Publishing, Cham, 234–241.
[28]
Abhijit Guha Roy, Nassir Navab, and Christian Wachinger. 2019. Recalibrating Fully Convolutional Networks With Spatial and Channel “Squeeze and Excitation” Blocks. IEEE Transactions on Medical Imaging 38, 2 (2019), 540–549. https://doi.org/10.1109/TMI.2018.2867261
[29]
David E. Rumelhart and James L. McClelland. 1986. Learning internal representations by error propagation. Parallel Distributed Processing 1 (1986), 318–363.
[30]
Axel Sauer, Katja Schwarz, and Andreas Geiger. 2022. StyleGAN-XL: Scaling StyleGAN to Large Diverse Datasets. In ACM SIGGRAPH 2022 Conference Proceedings (Vancouver, BC, Canada) (SIGGRAPH ’22). Association for Computing Machinery, New York, NY, USA, Article 49, 10 pages. https://doi.org/10.1145/3528233.3530738
[31]
Ramprasaath R Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. 2017. Grad-cam: Visual explanations from deep networks via gradient-based localization. In Proceedings of the IEEE international conference on computer vision. 618–626.
[32]
Wenzhe Shi, Jose Caballero, Ferenc Huszar, Johannes Totz, Andrew P. Aitken, Rob Bishop, Daniel Rueckert, and Zehan Wang. 2016. Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network. In CVPR. IEEE Computer Society, 1874–1883. http://dblp.uni-trier.de/db/conf/cvpr/cvpr2016.html#ShiCHTABRW16
[33]
Youssef Skandarani, Pierre-Marc Jodoin, and Alain Lalande. 2023. GANs for Medical Image Synthesis: An Empirical Study. Journal of Imaging 9, 3 (2023). https://doi.org/10.3390/jimaging9030069
[34]
Yi Sun, Peisen Yuan, and Yuming Sun. 2020. MM-GAN: 3D MRI Data Augmentation for Medical Image Segmentation via Generative Adversarial Networks. In 2020 IEEE International Conference on Knowledge Graph (ICKG). 227–234. https://doi.org/10.1109/ICBK50248.2020.00041
[35]
Albert Swiecicki, Nianyi Li, Jonathan O’Donnell, Nicholas Said, Jichen Yang, Richard C Mather, William A Jiranek, and Maciej A Mazurowski. 2021. Deep learning-based algorithm for assessment of knee osteoarthritis severity in radiographs matches performance of radiologists. Computers in biology and medicine 133 (2021), 104334.
[36]
Christian Szegedy, Sergey Ioffe, Vincent Vanhoucke, and Alexander A. Alemi. 2017. Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (San Francisco, California, USA) (AAAI’17). AAAI Press, 4278–4284.
[37]
Mingxing Tan and Quoc Le. 2019. EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks. In Proceedings of the 36th International Conference on Machine Learning(Proceedings of Machine Learning Research, Vol. 97), Kamalika Chaudhuri and Ruslan Salakhutdinov (Eds.). PMLR, 6105–6114. https://proceedings.mlr.press/v97/tan19a.html
[38]
Mingxing Tan and Quoc V. Le. 2021. EfficientNetV2: Smaller Models and Faster Training. In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event(Proceedings of Machine Learning Research, Vol. 139), Marina Meila and Tong Zhang (Eds.). PMLR, 10096–10106. http://proceedings.mlr.press/v139/tan21a.html
[39]
Aleksei Tiulpin, Jérôme Thevenot, Esa Rahtu, Petri Lehenkari, and Simo Saarakkala. 2018. Automatic knee osteoarthritis diagnosis from plain radiographs: a deep learning-based approach. Scientific reports 8, 1 (2018), 1727.
[40]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).
[41]
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In the 25th International Conference on Machine learning (ICML). 1096––1103.
[42]
Rima Tri Wahyuningrum, Lilik Anifah, I Ketut Eddy Purnama, and Mauridhi Hery Purnomo. 2016. A novel hybrid of S2DPCA and SVM for knee osteoarthritis classification. In 2016 IEEE International Conference on Computational Intelligence and Virtual Environments for Measurement Systems and Applications (CIVEMSA). IEEE, 1–5.
[43]
Jingdong Wang, Ke Sun, Tianheng Cheng, Borui Jiang, Chaorui Deng, Yang Zhao, Dong Liu, Yadong Mu, Mingkui Tan, Xinggang Wang, 2020. Deep high-resolution representation learning for visual recognition. IEEE transactions on pattern analysis and machine intelligence 43, 10 (2020), 3349–3364.
[44]
Zhe Wang, Aladine Chetouani, and Rachid Jennane. 2023. Transformer with Selective Shuffled Position Embedding using ROI-Exchange Strategy for Early Detection of Knee Osteoarthritis. arXiv preprint arXiv:2304.08364 (2023).
[45]
Di You, Shixin Wang, Futao Wang, Yi Zhou, Zhenqing Wang, Jingming Wang, and Yibing Xiong. 2022. EfficientUNet+: A Building Extraction Method for Emergency Shelters Based on Deep Learning. Remote Sensing 14, 9 (2022). https://doi.org/10.3390/rs14092207
[46]
Richard Zhang, Phillip Isola, Alexei A. Efros, Eli Shechtman, and Oliver Wang. 2018. The Unreasonable Effectiveness of Deep Features as a Perceptual Metric. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE Computer Society, Los Alamitos, CA, USA, 586–595. https://doi.org/10.1109/CVPR.2018.00068
[47]
Kang Zheng, Yirui Wang, Chen-I Hsieh, Le Lu, Jing Xiao, Chang-Fu Kuo, and Shun Miao. 2021. Coherence Learning using Keypoint-based Pooling Network for Accurately Assessing Radiographic Knee Osteoarthritis. (2021). arxiv:2112.09177 [eess.IV]
Index Terms
- OsteoGA: An Explainable AI Framework for Knee Osteoarthritis Severity Assessment
Recommendations
Automatic diagnosis of knee osteoarthritis severity using Swin transformer
CBMI '23: Proceedings of the 20th International Conference on Content-based Multimedia IndexingKnee osteoarthritis (KOA) is a widespread condition that can cause chronic pain and stiffness in the knee joint. Early detection and diagnosis are crucial for successful clinical intervention and management to prevent severe complications, such as loss ...
Clustering-Based Support Vector Machine (SVM) for Symptomatic Knee Osteoarthritis Severity Classification
ICBBE '22: Proceedings of the 2022 9th International Conference on Biomedical and Bioinformatics EngineeringThe application of machine learning in gait biomechanics of knee osteoarthritis (OA) had been reported in many previous studies. Instead of using radiographic severity classification to grade OA, this study employed pain scores and gait features to ...
Detection of Knee Osteoarthritis Stages Using Convolutional Neural Network
AbstractNowadays, many people suffer from the problem of knee osteoarthritis (KOA). As the age increases, the chances of suffering from knee osteoarthritis also increases simultaneously. Degeneration of the articular cartilage, the flexible, slick ...
Comments
Information & Contributors
Information
Published In
December 2023
1058 pages
ISBN:9798400708916
DOI:10.1145/3628797
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 07 December 2023
Check for updates
Author Tags
Qualifiers
- Research-article
- Research
- Refereed limited
Conference
SOICT 2023
SOICT 2023: The 12th International Symposium on Information and Communication Technology
December 7 - 8, 2023
Ho Chi Minh, Vietnam
Acceptance Rates
Overall Acceptance Rate 147 of 318 submissions, 46%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 70Total Downloads
- Downloads (Last 12 months)43
- Downloads (Last 6 weeks)1
Reflects downloads up to 01 Mar 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderHTML Format
View this article in HTML Format.
HTML Format