Abstract
Colorectal cancer, a formidable health hazard, necessitates the development of innovative and accurate diagnostic instruments in light of the rising mortality rates associated with gastrointestinal disorders. The introduction of deep learning algorithms has revolutionised disease detection, but the search for cutting-edge techniques continues to be essential. Enter GIEnsemformerCADx, an innovative hybrid approach poised to revolutionise early colorectal cancer detection. This diagnostic juggernaut offers a comprehensive solution by combining the formidable capabilities of vision transformers, fusion CNNs, and bidirectional LSTM models. Vision transformers derive high-level features from transformed data representations, whereas Fusion CNNs interpret complex spatial correlations within input images. The bidirectional LSTM model complements these advantages by enhancing the understanding of temporal relationships, resulting in an accurate and timely diagnosis of colorectal cancer. The Hyper Kvasir dataset was meticulously calibrated and rebalanced for training purposes, resulting in an optimised training corpus consisting of nine classes extracted from the original 23. The ten-class mixed CKHK-22 dataset was then subjected to rigorous evaluation, confirming the reliability of this method. Using well-known CNN architectures, such as AlexNet, DarkNet-19, ResNet-50, and DenseNet-201, within the CADx system, novel CNN fusion models (ADaDR-22, ADaR-22, and DaRD-22) were created by fusing these pre-trained CNNs. In identifying colorectal cancer, the DaRD-22 model outperformed its competitors, with a remarkable accuracy rate of 93.3% for Hyper Kvasir and 91.67% for the CKHK-22 datasets. GIEnsemformerCADx represents a major advancement in computer-aided colorectal cancer detection. Utilizing hybrid innovation and propelled by the exceptional performance of the DaRD-22 model, it promises to improve patient outcomes and reduce mortality rates through early detection and prompt intervention. In the ever-present battle against colorectal cancer, this innovative system is a beacon of hope and progress.
Similar content being viewed by others
Data availability
This study utilises data from the CVC clinic DB, Kvasir 2, and Hyper Kvasir datasets, as well as other publicly accessible colonoscopy databases, to advance medical knowledge and improve healthcare. Notably, although information about internal human organs is available on the Internet, conducting research in this sensitive area is subject to stringent ethical scrutiny by the relevant authorities. This research's data was obtained from reputable sources to assure its integrity and conformity with ethical standards. The CVC clinic DB dataset was downloaded from Kaggle, the Kvasir2 dataset was downloaded from datasets.simula.no, and the Hyper Kvasir dataset was also downloaded from datasets.simula.no. These publicly accessible datasets are indispensable for scientific research, promoting advancements in the comprehension and diagnosis of medical conditions such as colorectal cancer.
Code availability
All of our experiments and data collecting were performed in the secure Google Co-Lab setting. It should be noted, however, that the study's code has not been made publicly accessible or shared via internet repositories.
References
Mathur P, Sathishkumar K, Chaturvedi M, Das P, Sudarshan KL, Santhappan S (2020) Cancer Statistics, 2020: Report From National Cancer Registry Programme, India. JCO Global Oncol-An Am Soc Clin Oncol J 6:1063–1075
"Americal cancer Society," ACS, [Online]. Available: https://www.cancer.org/cancer/colon-rectal-cancer/. Accessed 15 Nov 2022
American Cancer Society (2020) Colorectal cancer facts & figures 2020-2022. American Cancer Society, Atlanta, GA
Kudo SE, Mori Y, Abdel-Aal UM, Misawa M, Itoh H, Oda M, Mori K (2021) Artificial intelligence and computer-aided diagnosis for colonoscopy: where do we stand now? Trans Gastroenterol Hepatol 6(64):25
Mitsala A, Tsalikidis C, Pitiakoudis M, Simopoulos C, Tsaroucha AK (2021) Artificial Intelligence in Colorectal Cancer Screening, Diagnosis and Treatment. A New Era. Curr Oncol 28(3):1581–1607
Fonollà R, van der Zander EWQ, Schreuder RM, Masclee AA, Schoon EJ, van der Sommen F, de With PH (2020) A CNN CADx System for Multimodal Classification of Colorectal Polyps Combining WL, BLI, and LCI Modalities. Appl Sci 10(15):5040
Raju AS, Jayavel K, Rajalakshmi T (2022) Intelligent recognition of colorectal cancer combining application of computer-assisted diagnosis with deep learning approaches. Int J Elect Comput Eng 12(1):738–747
Wang YK, Syu HY, Chen YH, Chung CS, Tseng YS, Ho SY, Huang CW, Wu IC, Wang HC (2021) Endoscopic Images by a Single-Shot Multibox Detector for the Identification of Early Cancerous Lesions in the Esophagus: A Pilot Study. Cancers 13(2):321
Banik D, Roy K, Bhattacharjee D, Nasipuri M, Krejcar O (2020) Polyp-Net: A Multimodel Fusion Network for Polyp Segmentation. IEEE Trans Instrum Meas 70:1–12
Öztürk Ş, Özkaya U (2021) Residual LSTM layered CNN for classification of gastrointestinal tract diseases. J Biomed Inform 113:103638
Meng Y, Zhang H, Zhao Y, Yang X, Qiao Y, MacCormick IJ, Huang X, Zheng Y (2022) Graph-Based Region and Boundary Aggregation for Biomedical Image Segmentation. IEEE Trans On Med Imaging 41(3):690–701
Park KB, Lee JY (2022) SwinE-Net: hybrid deep learning approach to novel polyp segmentation using convolutional neural network and Swin Transformer. J Comput Des Eng 9(2):616–632
Duc NT, Oanh NT, Thuy NT, Triet TM, Dinh VS (2022) ColonFormer: An Efficient Transformer based Method for Colon Polyp Segmentation. IEEE Access 10:80575–80586
Li K, Fathan MI, Patel K, Zhang T, Zhong C, Bansal A et al (2021) Colonoscopy polyp detection and classification: dataset creation and comparative evaluations. PLoS ONE 16(8):e0255809. https://doi.org/10.1371/journal.pone.0255809
Akella S. Narasimha Raju, Kayalvizhi Jayavel, T. Rajalakshmi, "ColoRectalCADx: Expeditious Recognition of Colorectal Cancer with Integrated Convolutional Neural Networks and Visual Explanations Using Mixed Dataset Evidence," Computational and Mathematical Methods in Medicine, 10 November 2022.
(2015) [Online]. https://www.kaggle.com/balraj98/cvcclinicdb. Accessed 25 May 2021
(2016) [Online]. https://datasets.simula.no/kvasir/. Accessed 3 July 2021
(2020) [Online]. https://datasets.simula.no/hyper-kvasir/. Accessed 3 July 2021
Cao R, Tang Z, Liu C, Veeravalli B (2020) A Scalable Multicloud Storage Architecture for Cloud-Supported Medical Internet of Things. IEE Int Things J 7(3):1641–1654
Oza P, Sharma P, Patel S, Adedoyin F, Bruno A (2022) A Image Augmentation Techniques for Mammogram Analysis. J Imaging 8(5):141
Chen CF, Fan Q, Panda R (2021) CrossViT: cross-attention multi-scale vision transformer for image classification. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, Montreal, QC, Canada, pp 347–356. https://doi.org/10.1109/ICCV48922.2021.00041
"https://towardsdatascience.com/," towards datascience, 5 October 2022. [Online]. Available: https://towardsdatascience.com/using-transformers-for-computer-vision-6f764c5a078b. [Accessed 5 June 2023].
Rahim T, Hassan SA, Shin SY (2021) A deep convolutional neural network for the detection of polyps in colonoscopy images. Biomed Signal Process Control 68:102654
Narasimha Raju AS, Jayavel K, Rajalakshmi T (2022) Dexterous Identification of Carcinoma through ColoRectalCADx with Dichotomous Fusion CNN and UNet Semantic Segmentation. Comput Intell Neurosci 2022:29
Igarashi S, Sasaki Y, Mikami T, Sakuraba H, Fukuda S (2020) Anatomical classification of upper gastrointestinal organs under various image capture conditions using AlexNet. Comput Biol Med 124:103950
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE, Honolulu, pp 7263–7271. https://doi.org/10.1109/CVPR.2017.690
Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, USA, pp 2261–2269. https://doi.org/10.1109/CVPR.2017.243
Szegedy C, Ioffe S, Vanhoucke V, Alemi A (2016) Inception-v4, inception-ResNet and the impact of residual connections on learning. In: Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, San Francisco, CA, pp 4278–4284
Tripathi S, Singh SK, Lee HK (2020) An end-to-end breast tumour classification model using context-based patch modelling – A BiLSTM approach for image classification. Comput Med Imaging Graph 87:101838
Wu H, Huang Q, Wang D, Gao L (2018) A CNN-SVM combined model for pattern recognition of knee motion using mechanomyography signals. J Electromyogr Kinesiol 42:136–142
Wu W, Li D, Du J, Gao X, Gu W, Zhao F, Feng X, Yan H (2020) An Intelligent Diagnosis Method of Brain MRI Tumor Segmentation Using Deep Convolutional Neural Network and SVM Algorithm. Comput Math Methods Med 2020:1–10
Attallah O, Sharkas M (2021) GASTRO-CADx: a three stages framework for diagnosing gastrointestinal diseases. PeerJ Comput Sci 7:e423
Liew WS, Tang TB, Lu CK (2022) Computer-aided diagnostic tool for classification of colonic polyp assessment. In: Ibrahim R, Porkumaran K, Kannan R, Mohd Nor N, Prabakar S (eds) International Conference on Artificial Intelligence for Smart Community. Lecture Notes in Electrical Engineering, vol 758. Springer, Singapore. https://doi.org/10.1007/978-981-16-2183-3_71
Sharma P, Balabantaray BK, Bora K, Mallik S, Kasugai K, Zhao Z (2022) An Ensemble-Based Deep Convolutional Neural Network for Computer-Aided Polyps Identification From Colonoscopy. Front Gen 13:844391
Nisha JS, Gopi VP, Palanisamy P (2022) Automated colorectal polyp detection based on image enhancement and dual-path CNN architecture. Biomed Signal Process Cont 73:103465
Raju AS, Jayavel K, Rajalakshmi T (2023) An advanced diagnostic ColoRectalCADx utilises CNN and unsupervised visual explanations to discover malignancies. Neural Comput Appl 35:20631–20662
Raju AS, Venkatesh K (2023) EnsemDeepCADx: Empowering Colorectal Cancer Diagnosis with Mixed-Dataset Features and Ensemble Fusion CNNs on Evidence-Based CKHK-22 Dataset. Bioengineering 10:738
Acknowledgements
We would like to express our deepest appreciation to the SRM Institute of Science and Technology in Kattankalattur, Chennai, for their unending enthusiasm and the use of their extensive resources during the course of our endeavour. Their dedication to research advancement and provision of state-of-the-art facilities, including high-tech hardware and software systems, has greatly enhanced our efforts. We would also like to thank the Institute of Aeronautical Engineering in Dundigal in Hyderabad for their support and assistance with our study. Their hard work has been crucial in creating a positive climate for research, especially in the field of deep learning-supported systems. We owe them a great debt of gratitude for helping us in our quest for scientific greatness and for their unwavering support.
Funding
All of the funding for this endeavor comes from inside the organization. The SRM Institute of Science and Technology in Kattanlkalatur has awarded a fellowship to one of their own in order to fund the whole of this endeavor.
Author information
Authors and Affiliations
Contributions
Significant contributions were made to this study by all four authors. Mr. Akella S. Narasimha Raju was responsible for assembling all datasets, running all tests, doing all analyses, and writing all of the original documentation. Dr. K. Venkatesh oversaw the project, analysed the results of the data, and wrote a portion of the article. Dr. G. Sucharitha Reddy curated the findings and did further analysis, while Dr. B. Padmaja came up with the study approach. The success of the research may be attributed in large part to the authors' ability to work together.
Corresponding author
Ethics declarations
Ethics approval
Although human colonoscopy images are utilised in this work, it is vital to keep in mind that the data used is not collected in real time. The necessary datasets for this investigation may be obtained for free from a wide variety of locations on the internet. Since this study is not collecting data in real time from human participants, the normal ethical clearance procedure for such investigations is unnecessary.
Consent to participate
Mr. Akella S. Narasimha Raju spearheaded the procurement of datasets from online sources, conducted meticulous testing, conducted in-depth data analysis, and crafted the initial documentation for this formidable collaboration. Dr. K. Venkatesh, the study's visionary, orchestrated the overall conceptualization, oversaw the intricate analysis of data outcomes, and played a crucial role in the documentation of the extensive research findings in this paper. Dr. B. Padmaja made a significant contribution by formulating the methodology, while Dr. G. Sucharitha Reddy curated the results and subjected them to rigorous analysis. This harmonious synthesis of evolving ideologies and devoted contributions propelled the research to its pinnacle.
Conflicts of interest
Mr. Akella S. Narasimha Raju, Dr. K. Venkatesh, Dr. B. Padmaja, and Dr. G. Sucharitha Reddy state that they have no financial stake in the outcome of this research. No author has any financial or other investment in the results of the research. The primary motivation for this study's existence is to further academic and institutional goals.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Raju, A.S.N., Venkatesh, K., Padmaja, B. et al. GIEnsemformerCADx: A hybrid ensemble learning approach for enhanced gastrointestinal cancer recognition. Multimed Tools Appl 83, 46283–46323 (2024). https://doi.org/10.1007/s11042-024-18521-4
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-024-18521-4