Abstract
Artificial intelligence (AI) is playing a growing role today in several areas, especially in health, where understanding AI models and their predictions is extremely important for health professionals. In this context, Explainable AI (XAI) plays a crucial role in seeking to provide understandable explanations for these models.
This article analyzes two different XAI approaches applied to analyzing gastric endoscopy images. The first, more conventional approach uses Grad CAM, while the second, even less explored but with great potential, is based on “similarity-based explanations”. This example-based XAI technique aims to provide representative examples to support the decisions of AI models.
In this study, we compare these two techniques applied to two different models: one based on the VGG16 architecture and the other based on ResNet50, designed to classify images from the KVASIR-capsule database. The results reveal that Grad-CAM provided intuitive explanations only for the VGG16 model, while the “similarity-based explanations” technique provided consistent explanations for both models. We conclude that exploring other XAI techniques can be a significant asset in improving the understanding of the various AI models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Maier, A., Syben, C., Lasser, T., Riess, C.: A gentle introduction to deep learning in medical image processing (2019). https://doi.org/10.1016/j.zemedi.2018.12.003
Do, S., Song, K.D., Chung, J.W.: Basics of deep learning: a radiologist’s guide to understanding published radiology articles on deep learning (2020). https://doi.org/10.3348/kjr.2019.0312
Barredo Arrieta, A., et al.: Explainable Artificial Intelligence (XAI): concepts, taxonomies, opportunities and challenges toward responsible AI (2020). https://doi.org/10.1016/j.inffus.2019.12.012
Huff, D.T., Weisman, A.J., Jeraj, R.: Interpretation and visualization techniques for deep learning models in medical imaging (2021). https://doi.org/10.1088/1361-6560/abcd17
Patrício, C., Neves, J.C., Teixeira, L.F.: Explainable deep learning methods in medical imaging diagnosis: a survey (2022). https://doi.org/10.48550/arxiv.2205.04766
Wang, S., Xing, Y., Zhang, L., Gao, H., Zhang, H.: Deep convolutional neural network for ulcer recognition in wireless capsule endoscopy: experimental feasibility and optimization (2019). https://doi.org/10.1155/2019/7546215
Malhi, A., Kampik, T., Pannu, H., Madhikermi, M., Framling, K.: Explaining machine learning-based classifications of in-vivo gastral images (2019). https://doi.org/10.1109/dicta47822.2019.8945986
Wickstrom, K., Kampffmeyer, M., Jenssen, R.: Uncertainty modeling and interpretability in convolutional neural networks for polyp segmentation (2018). https://doi.org/10.1109/mlsp.2018.8516998
Lima, D.L.S., Pessoa, A.C.P., De Paiva, A.C., da Silva Cunha, A.M.T., Júnior, G.B., De Almeida, J.D.S.: Classification of video capsule endoscopy images using visual transformers. In: 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI), pp. 1–4 (2022). https://doi.org/10.1109/BHI56158.2022.9926791
Fonseca, F., Nunes, B., Salgado, M., Cunha, A.: Abnormality classification in small datasets of capsule endoscopy images. Proc. Comput. Sci. 196, 469–476 (2022). https://doi.org/10.1016/j.procs.2021.12.038
Gomes, S., Valério, M.T., Salgado, M., Oliveira, H.P., Cunha, A.: Unsupervised neural network for homography estimation in capsule endoscopy frames. Proc. Comput. Sci. 164, 602–609 (2019). https://doi.org/10.1016/j.procs.2019.12.226
Smedsrud, P.H., et al.: Kvasir-capsule, a video capsule endoscopy dataset (2021). https://doi.org/10.1038/s41597-021-00920-z
Simonyan, K., Zisserman, A.: Very Deep Convolutional Networks for Large-Scale Image Recognition (2014). https://doi.org/10.48550/arxiv.1409.1556
He, K., Zhang, X., Ren, S., Sun, J.: Deep Residual Learning for Image Recognition (2016). https://doi.org/10.1109/cvpr.2016.90
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: Visual Explanations From Deep Networks via Gradient-Based Localization (2017). https://doi.org/10.1109/iccv.2017.74
Hanawa, K., Yokoi, S., Hara, S., Inui, K.: Evaluation of Similarity-Based Explanations (2020). https://doi.org/10.48550/arxiv.2006.04528
Charpiat, G., Girard, N., Felardos, L., Tarabalka, Y.: Input Similarity From the Neural Network Perspective (2021). https://doi.org/10.48550/arxiv.2102.05262
Acknowledgements
This work is financed by National Funds through the Portuguese funding agency,FCT Fundacäo para a Ciéncia e a Tecnologia, within project PTDC/EEIEEE/5557/2020. Co funded by the European Union (grant number 101095359) and supported by the UK Research and Innovation (grant number 10058099). Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the Health and Digital Executive Agency (HaDEA). Neither the European Union nor the granting authority can be held responsible for them.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Fontes, M., Leite, D., Dallyson, J., Cunha, A. (2024). Similarity-Based Explanations for Deep Interpretation of Capsule Endoscopy Images. In: Cunha, A., Paiva, A., Pereira, S. (eds) Wireless Mobile Communication and Healthcare. MobiHealth 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 578. Springer, Cham. https://doi.org/10.1007/978-3-031-60665-6_16
Download citation
DOI: https://doi.org/10.1007/978-3-031-60665-6_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-60664-9
Online ISBN: 978-3-031-60665-6
eBook Packages: Computer ScienceComputer Science (R0)