Abstract
Numerous applications have explored medical image classification using deep learning models. With the emergence of Explainable AI (XAI), researchers have begun to recognize its potential in validating the authenticity and correctness of results produced by black-box deep learning models. On the other hand, current diagnostic approaches for osteonecrosis face significant challenges, including difficulty in early detection, subjectivity in image interpretation, and reliance on surgical interventions without a comprehensive diagnostic foundation. This paper presents a novel Medical Computer-Aid-Diagnosis System—the Shadow Learning System framework—which integrates a convolutional neural network (CNN) with an Explainable AI method. This system not only performs conventional computer-aiding-diagnosis functions but also uniquely exploits misclassified data samples to provide additional medically relevant information from the machine learning model’s perspective, assisting doctors in their diagnostic process. The implementation of XAI techniques in our proposed system goes beyond merely validating CNN model results; it also enables the extraction of valuable information from medical images through an unconventional machine learning perspective. Our paper aims to enhance and extend the general structure and detailed design of the Shadow Learner System, making it more advantageous not only for human users but also for the deep learning model itself. A case study on femoral head osteonecrosis was conducted using our proposed system, which demonstrated improved accuracy and reliability in its prediction results. Experimental results interpreted using XAI methods are visualized to prove the confidence of our proposed model that generates reasonable results, confirming the effectiveness of the proposed model.
Graphical Abstract
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The code that support the findings of this study are available upon reasonable request to the corresponding author S.F at the corresponding email address.
Code availability
The code that support the findings of this study are available reasonable request to the corresponding author Y.W at the corresponding email address.
References
Zhao D, Zhang F, Wang B, Liu B, Li L, Kim SY, Goodman SB, Hernigou P, Cui Q, Lineaweaver WC, Xu J, Drescher WR, Qin L (2020) Guidelines for clinical diagnosis and treatment of osteonecrosis of the femoral head in adults (2019 version). J Orthop Translat 21:100–110. https://doi.org/10.1016/j.jot.2019.12.004
Seeram E (2023) X-Ray imaging systems: an overview. In: X-Ray Imaging Systems for Biomedical Engineering Technology. Springer, Cham. pp 1–15. https://doi.org/10.1007/978-3-031-46266-5_1
Sarwani NE, Gardner JA (2023) Computed Tomography (CT) Scan Basics. In: Docimo S Jr, Blatnik JA, Pauli EM (eds) Fundamentals of Hernia Radiology. Springer, Cham. https://doi.org/10.1007/978-3-031-21336-6_1
Bruno F, Granata V, CobianchiBellisari F, Sgalambro F, Tommasino E, Palumbo P, Arrigoni F, Cozzi D, Grassi F, Brunese MC, Pradella S, di SS tefano MLM, Cutolo C, Di Cesare E, Splendiani A, Giovagnoni A, Miele V, Grassi R, Masciocchi C, Barile A (2022) Advanced Magnetic Resonance Imaging (MRI) Techniques: Technical Principles and Applications in Nanomedicine. Cancers (Basel) 14(7):1626. https://doi.org/10.3390/cancers14071626
Naser MA, Deen MJ (2020) Brain tumor segmentation and grading of lower-grade glioma using deep learning in MRI images. Comput Biol Med 121:103758. https://doi.org/10.1016/j.compbiomed.2020.103758
Wang C et al (2021) Deep learning for predicting subtype classification and survival of lung adenocarcinoma on computed tomography. Transl Oncol 14(8):101141. https://doi.org/10.1016/j.tranon.2021.101141
Iqbal S et al (2021) Prostate cancer detection using deep learning and traditional techniques. IEEE Access 9:27085–27100. https://doi.org/10.1109/ACCESS.2021.3057654
Doi K (2007) Computer-aided diagnosis in medical imaging: historical review, current status and future potential. Comput Med Imaging Graph 31(4–5):198–211. https://doi.org/10.1016/j.compmedimag.2007.02.002
Liu L, Feng W, Chen C, Liu M, Qu Y, Yang J (2022) Classification of breast cancer histology images using MSMV-PFENet. Sci Rep 12(1):17447
Lei Y, Tian Y, Shan H, Zhang J, Wang G, Kalra MK (2020) Shape and margin-aware lung nodule classification in low-dose CTimages via soft activation mapping. Med Image Anal 60:101628
Yaoyang WU, Simon FONG, Liansheng LIU (2024) “Shadow Learner System: Implementing a Convolutional Neural Network with Explainable AI for Enhanced Bone Radiology Image Classification”, Soft Computing, Accepted
Zhou B, Khosla A, Lapedriza A, Oliva A. and Torralba A. (2016) Learning deep features for discriminative localization. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) pp 2921–2929. https://doi.org/10.1109/CVPR.2016.319.
Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-CAM: visual explanations from deep networks via gradient-based localization. IEEE International Conference on Computer Vision (ICCV) pp 618–626. https://doi.org/10.1109/ICCV.2017.74.
Kadir T, Brady M (2001) Saliency, scale and image description. Int J Comput Vision 45(2):83–105
Li Z, Guo C, Nie D, Lin D, Zhu Yi, Chen C, Xiang Y, Xu F, Jin C, Zhang X, Yang Y, Zhang K, Zhao L, Zhang P, Han Yu, Yun D, Wu X, Yan P, Lin H (2020) Development and Evaluation of a Deep Learning System for Screening Retinal Hemorrhage Based on Ultra-Widefield Fundus Images. Transl Vision Sci Technol 9:3. https://doi.org/10.1167/tvst.9.2.3
Bien N, Rajpurkar P, Ball RL, Irvin J, Park A, Jones E, Bereket M, Patel BN, Yeom KW, Shpanskaya K, Halabi S, Zucker E, Fanton G, Amanatullah DF, Beaulieu CF, Riley GM, Stewart RJ, Blankenberg FG, Larson DB, Jones RH, Langlotz CP, Ng AY, Lungren MP (2018) Deep-learning-assisted diagnosis for knee magnetic resonance imaging: Development and retrospective validation of MRNet. PLoS Med 15(11):e1002699. https://doi.org/10.1371/journal.pmed.1002699
Dunnmon JA, Yi D, Langlotz CP, Ré C, Rubin DL, Lungren MP (2019) Assessment of convolutional neural networks for automated classification of chest radiographs. Radiology. 290(2):537–544. https://doi.org/10.1148/radiol.2018181422
Larson DB, Chen MC, Lungren MP, Halabi SS, Stence NV, Langlotz CP (2018) Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology 287(1):313–322. https://doi.org/10.1148/radiol.2017170236
Grassmann F, Mengelkamp J, Brandl C, Harsch S, Zimmermann ME, Linkohr B, Peters A, Heid IM, Palm C, Weber BHF (2018) A deep learning algorithm for prediction of age-related eye disease study severity scale for age-related macular degeneration from color fundus photography. Ophthalmology 125(9):1410–1420. https://doi.org/10.1016/j.ophtha.2018.02.037
Dubost F, Adams H, Bortsova G, Ikram MA, Niessen W, Vernooij M, de Bruijne M (2019Jan) 3D regression neural network for the quantification of enlarged perivascular spaces in brain MRI. Med Image Anal 51:89–100. https://doi.org/10.1016/j.media.2018.10.008
Magesh PR, Myloth RD, Tom RJ (2020) An Explainable Machine Learning Model for Early Detection of Parkinson’s Disease using LIME on DaTSCAN Imagery. Comput Biol Med 126:104041. https://doi.org/10.1016/j.compbiomed.2020.104041. (ISSN 0010-4825)
Ribeiro MT, Singh S, Guestrin C (2016) Why should i trust you? Explaining the predictions of any classifier. 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD’16). pp 1135–1144. https://doi.org/10.1145/2939672.2939778
Yang C, Rangarajan A, Ranka S (2018) Visual explanations from deep 3D convolutional neural networks for alzheimer’s disease classification. AMIA Annu Symp Proc 2018:1571–1580
de Sousa Palatnik I, Maria-Bernardes-Rebuzzi-Vellasco M, da Silva Costa E (2019) Local Interpretable Model-Agnostic Explanations for Classification of Lymph Node Metastases. Sensors 19:2969. https://doi.org/10.3390/s19132969
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely Connected Convolutional Networks. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 2017:2261–2269. https://doi.org/10.1109/CVPR.2017.243
He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 770–778. https://doi.org/10.1109/CVPR.2016.90
Szegedy C, Ioffe S, Vanhoucke Vi, Alemi A (2017) Inceptionv4, inception-resnet and the impact of residual connections on learning. AAAI Conference on Artificial Intelligence (AAAI’17). pp 4278–4284. https://doi.org/10.1609/aaai.v31i1.11231
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016). Rethinking the inception architecture for computer vision. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 2016: 2818-2826. https://doi.org/10.1109/CVPR.2016.308
Szegedy C et al. (2015) Going deeper with convolutions. IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp 1–9. https://doi.org/10.1109/CVPR.2015.7298594
Jiang P-T, Zhang C-B, Hou Q, Cheng M-M, Wei Y (2021) Layer-CAM: exploring hierarchical class activation maps for localization. IEEE Transactions on Image Processing (TIP’21). 30:5875–5888. https://doi.org/10.1109/TIP.2021.3089943
Raghu M, Zhang C, Kleinberg JM, Bengio S (2019) Transfusion: understanding transfer learning for medical imaging. 33rd International Conference on Neural Information Processing Systems. 301:3347–3357. https://doi.org/10.5555/3454287.3454588
Schäfer R, Nicke T, Höfener H et al (2024) Overcoming data scarcity in biomedical imaging with a foundational multi-task model. Nat Comput Sci 4:495–509. https://doi.org/10.1038/s43588-024-00662-z
Mei X et al (2022) RadImageNet: an open radiologic deep learning research dataset for effective transfer learning. Radiol Artif Intell 4:e210315
Zhou H-Y et al (2022) Generalized radiograph representation learning via cross-supervision between images and free-text radiology reports. Nat Mach Intell 4:32–40
Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2 (NIPS'14). MIT Press, Cambridge, MA, USA, 3320–3328
Rajpurkar P, Irvin J, Bagul A, Ding D, Duan T, Mehta H, Yang B, Zhu K, Laird D, Ball R, Langlotz C, Shpanskaya K, Lungren, M, Ng, A (2017) MURA Dataset: Towards Radiologist-Level Abnormality Detection in Musculoskeletal Radiographs
Chen P (2018) Knee Osteoarthritis Severity Grading Dataset, Mendeley Data, V1, https://doi.org/10.17632/56rmx5bjcr.1
Štajduhar I, Mamula M, Miletić D, Uenal G (2017) Semi-automated detection of anterior cruciate ligament injury from MRI. Comput Methods Programs Biomed 140:151–164
Kapishnikov A, Bolukbasi T, Viegas F, Terry M (2019) XRAI: Better Attributions Through Regions," 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 4947-4956, https://doi.org/10.1109/ICCV.2019.00505
Adebayo J, Gilmer J, Muelly M, Goodfellow I, Hardt M, Kim B (2018) Sanity checks for saliency maps. In Proceedings of the 32nd International Conference on Neural Information Processing Systems (NIPS'18). Curran Associates Inc., Red Hook, NY, USA, 9525–9536
Abedeen I, Rahman MA, Prottyasha FZ et al (2023) FracAtlas: A dataset for fracture classification, localization and segmentation of musculoskeletal radiographs. Sci Data 10:521. https://doi.org/10.1038/s41597-023-02432-4
Buslaev A, Iglovikov V, Khvedchenya E, Parinov A, Druzhinin M, Kalinin A (2020) Albumentations: Fast and flexible image augmentations. Information 11:125. https://doi.org/10.3390/info11020125
Kolesnyk A, Khairova N (2022) Justification for the use of cohen’s kappa statistic in experimental studies of NLP and text mining. Cybern Syst Anal 58:280–288. https://doi.org/10.1007/s10559-022-00460-3
Sahin ME (2023) Image processing and machine learning-based bone fracture detection and classification using X-ray images. Int J Imaging Syst Technol 33(3):853–865. https://doi.org/10.1002/ima.22849
Rao LJ, Neelakanteswar P, Ramkumar M, Krishna A, Basha CZ (2020) An Effective Bone Fracture Detection using Bag-of-Visual Words with the Features Extracted from SIFT. In International Conference on Electronics and Sustainable Communication Systems (ICESC), pp. 6–10. doi-link
Funding
This project was supported by grants from the National Key R&D Program of China (2021YFE0201100 and 2022YFA1103401 to J.G.); National Natural Science Foundation of China (981890991 to J.G.), Beijing Municipal Natural Science Foundation (Z200021 to J.G.); CAS Interdisciplinary Innovation Team (JCTD-2020–04 to J.G.); 0032/2022/A, by Macau FDCT, MYRG2022-00271-FST by University of Macau, and Guangzhou Development Zone Science and Technology Project (2021GH10).
Author information
Authors and Affiliations
Contributions
Y.W: Conceptualization, Methodology, Software, Writing- Original draft preparation S.F: Conceptualization, Supervision, Writing- Reviewing and Editing L.L: Data curation, Supervision, Reviewing and Editing. L.L contributed to Data Curation, Formal Analysis, Validation, Writing- Reviewing and Editing.
Corresponding author
Ethics declarations
Declarations
All authors confirm that they had full access to all the data in the study and accept responsibility to submit for publication. Y.W and S.F are from the academic team of University of Macau.
Ethics approval
This study is approved by the Human Ethics Committee of The Sixth Affiliated Hospital of Jinan University, Guangdong, China.
Consent to participate
Written informed consent was obtained from individual or guardian participants.
Consent for publication
Not applicable.
Conflicts of interest
The authors declare that there is no conflict of interests regarding the publication of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Wu, Y., Fong, S. & Liu, L. Enhancing explainability in medical image classification and analyzing osteonecrosis X-ray images using shadow learner system. Appl Intell 55, 137 (2025). https://doi.org/10.1007/s10489-024-05916-x
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-05916-x