Abstract
Automatic and accurate analysis of medical images is a subject of great importance in our current society. In particular, this work focuses on gastrointestinal endoscopy images, as the study of these images helps to detect possible health conditions in those regions. Published works on this topic mainly used traditional classification methods (e.g., Support Vector Machines) or more modern techniques, such as Convolutional Neural Networks. However, little attention has been paid to more recent approaches such as Transformers or, in general, Attention-based Deep Neural Networks. This work aims to evaluate the performance of state-of-the-art attention-based models on the problem of classification of gastrointestinal endoscopy images. The experimental results on the challenging Hyper-Kvasir dataset indicate that attention-based models achieve performance equal to or better than that obtained by previous models, needing fewer parameters. In addition, a new state of the art on Hyper-Kvasir (i.e., 0.636 F1-Macro) is obtained by the fusion of two MobileViT models with only 20M parameters. The source code will be published here: https://github.com/richardesp/Attention-based-models-for-Hyper-Kvasir/.
Supported by projects TED2021-129151B-I00/AEI/10.13039/ 501100011033/European Union NextGenerationEU/PRTR and PID2019-103871GB-I00 of the Spanish Ministry of Economy, Industry and Competitiveness.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Base models: https://github.com/leondgarse/keras_cv_attention_models.
References
Compute class weight in sklearn utils. https://scikit-learn.org/stable/modules/generated/sklearn.utils.class_weight.compute_class_weight.html
Structural analysis and shape descriptors. https://docs.opencv.org/3.4/d3/dc0/group__imgproc__shape.html#ga17ed9f5d79ae97bd4c7cf18403e1689a
Borgli, H., Thambawita, V., Smedsrud, P.H., Hicks, S., et al.: Hyperkvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci. Data 7(1), 283 (2020)
Chen, P.J., Lin, M.C., Lai, M.J., Lin, J.C., et al.: Accurate classification of diminutive colorectal polyps using computer-aided analysis. Gastroenterology 154(3), 568–575 (2018)
Dai, Z., Liu, H., Le, Q.V., Tan, M.: Coatnet: marrying convolution and attention for all data sizes. In: NEURIPS, vol. 34 (2021)
Ding, M., Xiao, B., Codella, N., Luo, P., et al.: DaViT: dual attention vision transformers. In: Avidan, S., Brostow, G., Cissé, M., Farinella, G.M., Hassner, T. (eds.) ECCV 2022. LNCS, vol. 13684, pp. 74–92. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20053-3_5
Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., et al.: An image is worth 16x16 words: transformers for image recognition at scale. In: ICLR (2021)
Galdran, A., Carneiro, G., Ballester, M.A.G.: A hierarchical multi-task approach to gastrointestinal image analysis. In: ICPR Workshops and Challenges (2021)
Galdran, A., Carneiro, G., González Ballester, M.A.: Balanced-MixUp for highly imbalanced medical image classification. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12905, pp. 323–333. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87240-3_31
Gjestang, H.L., Hicks, S.A., Thambawita, V., Halvorsen, P., Riegler, M.A.: A self-learning teacher-student framework for gastrointestinal image classification. In: 2021 IEEE 34th International Symposium on CBMS (2021)
Guo, J., Han, K., Wu, H., Tang, Y., et al.: CMT: Convolutional neural networks meet vision transformers. In: IEEE/CVF CVPR (2022)
Hirasawa, T., Aoyama, K., Tanimoto, T., Ishihara, S., et al.: Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21, 653–660 (2018)
Mehta, S., Rastegari, M.: MobileViT: light-weight, general-purpose, and mobile-friendly vision transformer. In: ICLR (2022)
Pogorelov, K., Randel, K.R., Griwodz, C., Eskeland, S.L., et al.: Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of ACM on Multimedia Systems Conference, pp. 164–169 (2017)
Shen, D., Wu, G., Suk, H.I.: Deep learning in medical image analysis. Ann. Rev. Biomed. Eng. 19, 221–248 (2017)
Thambawita, V., Strümke, I., Hicks, S.A., Halvorsen, P., et al.: Impact of image resolution on deep learning performance in endoscopy image classification: an experimental study using a large dataset of endoscopic images. Diagnostics 11(12), 2183 (2021)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., et al.: Attention is all you need. In: NeurIPS, vol. 30 (2017)
Wu, J., Chen, X.Y., Zhang, H., Xiong, L.D., et al.: Hyperparameter optimization for machine learning models based on Bayesian optimization. J. Electron. Sci. Technol. 17(1), 26–40 (2019)
Yue, G., Wei, P., Liu, Y., Luo, Y., et al.: Automated endoscopic image classification via deep neural network with class imbalance loss. IEEE Trans. Instrum. Meas. 72, 1–11 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Espantaleón-Pérez, R., Jiménez-Velasco, I., Muñoz-Salinas, R., Marín-Jiménez, M.J. (2023). Empirical Study of Attention-Based Models for Automatic Classification of Gastrointestinal Endoscopy Images. In: Tsapatsoulis, N., et al. Computer Analysis of Images and Patterns. CAIP 2023. Lecture Notes in Computer Science, vol 14185. Springer, Cham. https://doi.org/10.1007/978-3-031-44240-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-44240-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44239-1
Online ISBN: 978-3-031-44240-7
eBook Packages: Computer ScienceComputer Science (R0)