Skip to main content
SpringerLink
Log in

Recognizing Gastrointestinal Malignancies on WCE and CCE Images by an Ensemble of Deep and Handcrafted Features with Entropy and PCA Based Features Optimization

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

In medical imaging, automated detection of stomach and gastrointestinal diseases using WCE (wireless capsule endoscopy) images is an emerging research domain. It includes numerous limitations and challenges such as variation in the contrast, texture variation, color and complexity in the background etc. To overcome these challenges, several computer-aided methods are proposed by the researchers. But there exist different limitations in these methods. In this work, a new method is proposed for computer-aided diagnosis of stomach disease classification. This hybrid approach is based on the amassed texture and deep CNN features. Initially, the contrast of image is improved by using power-law transformation. Texture features are extracted from the enhanced dataset by using LBP and SFTA features. Extracted texture features are then fused to obtain strong feature vectors. At the same time, two pre-trained deep learning models are utilized for CNN feature extraction namely VGG16 and InceptionV3. Extracted deep features are fused serially along with the obtained handcrafted feature vector to obtain an ensemble deep feature vector. A unique feature vector is obtained by serial fusion of both fused vectors to get an advantage of accumulated texture and CNN features. This feature vector is supplied to various classifiers and evaluated with existing methods. The promising recognition proficiency portrays the strength of proposed approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Bray F, Ferlay J, Soerjomataram I, Siegel RL, Torre LA, Jemal A (2018) Global cancer statistics 2018: GLOBOCAN estimates of incidence and mortality worldwide for 36 cancers in 185 countries. CA Cancer J Clin 68:394–424

    Google Scholar 

  2. Park SR, Kim MJ, Ryu KW, Lee JH, Lee JS, Nam B-H et al (2010) Prognostic value of preoperative clinical staging assessed by computed tomography in resectable gastric cancer patients: a viewpoint in the era of preoperative treatment. Ann Surg 251:428–435

    Google Scholar 

  3. A. C. Society. (2020, 30-07-2020). Cancer facts and figures 2020. Available: https://www.cancer.org/content/dam/cancer-org/research/cancer-facts-and-statistics/annual-cancer-facts-and-figures/2020/cancer-facts-and-figures-2020.pdf

  4. Statista. Prevalence of diagnosed gastrointestinal conditions in selected countries as of 2018. https://www.statista.com/statistics/418515/adult-prevalence-of-gastrointestinal-conditions-by-country/

  5. N. Master. (3-6-2019). Diseases of the digestive system deaths per 100,000 population (1995–1998). https://www.nationmaster.com/country-info/stats/Health/Digestive-disease-deaths

  6. Kim Y-W, Baik YH, Yun YH, Nam BH, Kim DH, Choi IJ et al (2008) Improved quality of life outcomes after laparoscopy-assisted distal gastrectomy for early gastric cancer: results of a prospective randomized clinical trial. Ann Surg 248:721–727

    Google Scholar 

  7. Liaqat A, Khan MA, Shah JH, Sharif M, Yasmin M, Fernandes SL (2018) Automated ulcer and bleeding classification from WCE images using multiple features fusion and selection. J Mech Med Biol 18:1850038

    Google Scholar 

  8. Asperti A, Mastronardo C (2017) The effectiveness of data augmentation for detection of gastrointestinal diseases from endoscopical images. arXiv preprint arXiv:1712.03689

  9. Yalamarthi S, Witherspoon P, McCole D, Auld C (2004) Missed diagnoses in patients with upper gastrointestinal cancers. Endoscopy 36:874–879

    Google Scholar 

  10. Iddan G, Meron G, Glukhovsky A, Swain P (2000) Wireless capsule endoscopy. Nature 405:417

    Google Scholar 

  11. Yuan Y, Wang J, Li B, Meng MQ-H (2015) Saliency based ulcer detection for wireless capsule endoscopy diagnosis. IEEE Trans Med Imaging 34:2046–2057

    Google Scholar 

  12. Yuan Y, Yao X, Han J, Guo L, Meng MQ-H (2017) Discriminative joint-feature topic model with dual constraints for WCE classification. IEEE Trans Cybern 48:2074–2085

    Google Scholar 

  13. Yuan Y, Meng MQ-H (2015) "Automatic bleeding frame detection in the wireless capsule endoscopy images. IEEE Int Conf Robot Autom 2015:1310–1315

    Google Scholar 

  14. Riaz F, Silva FB, Ribeiro MD, Coimbra MT (2012) Impact of visual features on the segmentation of gastroenterology images using normalized cuts. IEEE Trans Biomed Eng 60:1191–1201

    Google Scholar 

  15. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D et al (2017) Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on multimedia systems conference, pp 164–169

  16. Pogorelov K, Randel KR, de Lange T, Eskeland SL, Griwodz C, Johansen D et al (2017) Nerthus: a bowel preparation quality video dataset. In: Proceedings of the 8th ACM on multimedia systems conference, pp 170–174

  17. Sharif M, Tanvir U, Munir EU, Khan MA, Yasmin M (2018) Brain tumor segmentation and classification by improved binomial thresholding and multi-features selection. J Ambient Intell Human Comput 8:1–20

    Google Scholar 

  18. Bokhari F, Syedia T, Sharif M, Yasmin M, Fernandes SL (2018) Fundus image segmentation and feature extraction for the detection of glaucoma: a new approach. Curr Med Imaging Rev 14:77–87

    Google Scholar 

  19. Kiraly AP, Petkov K, Park JH (2019) Two-dimensional cinematic medical imaging in color based on deep learning. Google Patents

  20. Sharif M, Khan MA, Zahid F, Shah JH, Akram T (2019) Human action recognition: a framework of statistical weighted segmentation and rank correlation-based selection. Pattern Anal Appl 6:1–14

    Google Scholar 

  21. Sharif M, Khan S, Saba T, Raza M, Rehman A (2019) Improved video stabilization using SIFT-log polar technique for unmanned aerial vehicles. Int Conf Comput Inf Sci 2019:1–7

    Google Scholar 

  22. Irshad M, Muhammad N, Sharif M, Yasmeen M (2018) Automatic segmentation of the left ventricle in a cardiac MR short axis image using blind morphological operation. Eur Phys J Plus 133:148

    Google Scholar 

  23. Qin C, Schlemper J, Caballero J, Price AN, Hajnal JV, Rueckert D (2019) Convolutional recurrent neural networks for dynamic MR image reconstruction. IEEE Trans Med Imaging 38:280–290

    Google Scholar 

  24. Gong K, Guan J, Kim K, Zhang X, Yang J, Seo Y et al (2019) Iterative PET image reconstruction using convolutional neural network representation. IEEE Trans Med Imaging 38:675–685

    Google Scholar 

  25. Rashid M, Khan MA, Sharif M, Raza M, Sarfraz MM, Afza F (2018) Object detection and classification: a joint selection and fusion strategy of deep convolutional neural network and SIFT point features. Multimed Tools Appl 78:1–27

    Google Scholar 

  26. Biswas M, Kuppili V, Saba L, Edla D, Suri H, Cuadrado-Godia E et al (2019) State-of-the-art review on deep learning in medical imaging. Front Biosci (Landmark edition) 24:392–426

    Google Scholar 

  27. Yuan Y, Li B, Meng MQ-H (2016) WCE abnormality detection based on saliency and adaptive locality-constrained linear coding. IEEE Trans Autom Sci Eng 14:149–159

    Google Scholar 

  28. Li B, Meng MQ-H (2012) Tumor recognition in wireless capsule endoscopy images using textural features and SVM-based feature selection. IEEE Trans Inf Technol Biomed 16:323–329

    Google Scholar 

  29. Faigel DO, Cave DR (2008) Capsule endoscopy. Elsevier, Saunders

    Google Scholar 

  30. Yuan Y, Li B, Meng MQ-H (2015) Improved bag of feature for automatic polyp detection in wireless capsule endoscopy images. IEEE Trans Autom Sci Eng 13:529–535

    Google Scholar 

  31. Li B, Meng MQ-H (2012) Automatic polyp detection for wireless capsule endoscopy images. Expert Syst Appl 39:10952–10958

    Google Scholar 

  32. Hartmann D, Schmidt H, Bolz G, Schilling D, Kinzel F, Eickhoff A et al (2005) A prospective two-center study comparing wireless capsule endoscopy with intraoperative enteroscopy in patients with obscure GI bleeding. Gastrointest Endosc 61:826–832

    Google Scholar 

  33. Bchir O, Ismail M, AL-Aseem N (2018) Empirical comparison of visual descriptors for ulcer recognition in wireless capsule endoscopy video. Comput Sci Inf Technol 1:18

    Google Scholar 

  34. Bchir O, Ismail MMB, AlZahrani N (2019) Multiple bleeding detection in wireless capsule endoscopy. SIViP 13:121–126

    Google Scholar 

  35. Khan MA, Sharif M, Akram T, Yasmin M, Nayak RS (2019) Stomach deformities recognition using rank-based deep features selection. J Med Syst 43:329

    Google Scholar 

  36. Khan MA, Rashid M, Sharif M, Javed K, Akram T (2019) Classification of gastrointestinal diseases of stomach from WCE using improved saliency-based method and discriminant features selection. Multimed Tools Appl 78:27743–27770

    Google Scholar 

  37. Majid A, Khan MA, Yasmin M, Rehman A, Yousafzai A, Tariq U (2020) Classification of stomach infections: a paradigm of convolutional neural network along with classical features fusion and selection. Microsc Res Tech 83:562–576

    Google Scholar 

  38. Khan MA, Khan MA, Ahmed F, Mittal M, Goyal LM, Hemanth DJ et al (2020) Gastrointestinal diseases segmentation and classification based on duo-deep architectures. Pattern Recogn Lett 131:193–204

    Google Scholar 

  39. Alaskar H, Hussain A, Al-Aseem N, Liatsis P, Al-Jumeily D (2019) Application of convolutional neural networks for automated ulcer detection in wireless capsule endoscopy images. Sensors 19:1265

    Google Scholar 

  40. Diamantis DE, Iakovidis DK, Koulaouzidis A (2019) Look-behind fully convolutional neural network for computer-aided endoscopy. Biomed Signal Process Control 49:192–201

    Google Scholar 

  41. Aoki T, Yamada A, Aoyama K, Saito H, Tsuboi A, Nakada A et al (2019) Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointestinal Endosc 89:357–363

    Google Scholar 

  42. Hajabdollahi M, Esfandiarpoor R, Khadivi P, Soroushmehr S, Karimi N, Najarian K et al (2018) Segmentation of bleeding regions in wireless capsule endoscopy for detection of informative frames. arXiv preprint arXiv:1808.07746

  43. Lan L, Ye C, Wang C, Zhou S (2019) Deep convolutional neural networks for WCE abnormality detection: CNN architecture, region proposal and transfer learning. IEEE Access

  44. Lee JH, Kim YJ, Kim YW, Park S, Choi Y-I, Kim YJ et al (2019) Spotting malignancies from gastric endoscopic images using deep learning. Surg Endosc 33:1–8

    Google Scholar 

  45. Sharif M, Attique Khan M, Rashid M, Yasmin M, Afza F, Tanik UJ (2019) Deep CNN and geometric features-based gastrointestinal tract diseases detection and classification from wireless capsule endoscopy images. J Exp Theor Artif Intell 8:1–23

    Google Scholar 

  46. Ali H, Yasmin M, Sharif M, Rehmani MH (2018) Computer assisted gastric abnormalities detection using hybrid texture descriptors for chromoendoscopy images. Comput Methods Programs Biomed 157:39–47

    Google Scholar 

  47. Sivakumar P, Kumar BM (2018) A novel method to detect bleeding frame and region in wireless capsule endoscopy video. Clust Comput 22:1–7

    Google Scholar 

  48. Deeba F, Bui FM, Wahid KA (2020) Computer-aided polyp detection based on image enhancement and saliency-based selection. Biomed Signal Process Control 55:101530

    Google Scholar 

  49. Sundaram PS, Santhiyakumari N (2019) An enhancement of computer aided approach for colon cancer detection in WCE images using ROI based color histogram and SVM2. J Med Syst 43:29

    Google Scholar 

  50. Yasar A, Saritas I, Korkmaz H (2019) Computer-aided diagnosis system for detection of stomach cancer with image processing techniques. J Med Syst 43:99

    Google Scholar 

  51. Deeba F, Islam M, Bui FM, Wahid KA (2018) Performance assessment of a bleeding detection algorithm for endoscopic video based on classifier fusion method and exhaustive feature selection. Biomed Signal Process Control 40:415–424

    Google Scholar 

  52. Ghosh T, Fattah SA, Wahid KA (2018) CHOBS: Color histogram of block statistics for automatic bleeding detection in wireless capsule endoscopy video. IEEE J Transl Eng Health Med 6:1–12

    Google Scholar 

  53. Hirasawa T, Aoyama K, Tanimoto T, Ishihara S, Shichijo S, Ozawa T et al (2018) Application of artificial intelligence using a convolutional neural network for detecting gastric cancer in endoscopic images. Gastric Cancer 21:653–660

    Google Scholar 

  54. He J-Y, Wu X, Jiang Y-G, Peng Q, Jain R (2018) Hookworm detection in wireless capsule endoscopy images with deep learning. IEEE Trans Image Process 27:2379–2392

    MathSciNet  MATH  Google Scholar 

  55. Vimal S, Thiruvikraman P (2012) Automated image enhancement using power law transformations. Sadhana 37:739–745

    MathSciNet  Google Scholar 

  56. Ojala T, Pietikäinen M, Harwood D (1996) A comparative study of texture measures with classification based on featured distributions. Pattern Recogn 29:51–59

    Google Scholar 

  57. El-Henawy I, El Bakry HM, El Hadad HM (2016) Cattle identification using segmentation-based fractal texture analysis and artificial neural networks. Int J Electron Inform Eng 4:82–93

    Google Scholar 

  58. Saba T, Khan MA, Rehman A, Marie-Sainte SL (2019) Region extraction and classification of skin cancer: a heterogeneous framework of deep CNN features fusion and reduction. J Med Syst 43:289

    Google Scholar 

  59. Khan MA, Rubab S, Kashif A, Sharif MI, Muhammad N, Shah JH et al (2020) Lungs cancer classification from CT images: an integrated design of contrast based classical features fusion and selection. Pattern Recogn Lett 129:77–85

    Google Scholar 

  60. Rehman A, Khan MA, Mehmood Z, Saba T, Sardaraz M, Rashid M (2020) Microscopic melanoma detection and classification: a framework of pixel-based fusion and multilevel features reduction. Microsc Res Tech 83:410–423

    Google Scholar 

  61. Khan MA, Sharif MI, Raza M, Anjum A, Saba T, Shad SA (2019) Skin lesion segmentation and classification: a unified framework of deep neural network features fusion and selection. Expert Syst 8:e12497

    Google Scholar 

  62. Hearst MA et al (1998) Support vector machines. IEEE Intell Syst Appl 13(4):18–28

    Google Scholar 

  63. Jain U, Nathani K, Ruban N, Joseph Raj AN, Zhuang Z, Mahesh VGV (2018) Cubic SVM classifier based feature extraction and emotion detection from speech signals. In: 2018 International conference on sensor networks and signal processing (SNSP), Xi'an, China, pp 386–391. https://doi.org/10.1109/SNSP.2018.00081

  64. Mohammadrezaei M, Shiri ME, Rahmani AM (2018) Identifying fake accounts on social networks based on graph analysis and classification algorithms. Secur Commun Netw 20:5–16

    Google Scholar 

  65. Arboleda ER (2019) Comparing performances of data mining algorithms for classification of green coffee beans. Int J Eng Adv Technol 8(5):1563–1567

    Google Scholar 

  66. Breiman L, Friedman J, Olshen R, Stone C (1984) Classification and regression trees. CRC Press, Boca Raton

    MATH  Google Scholar 

  67. Kundu AK, Fattah SA, Wahid KA (2020) Least square saliency transformation of capsule endoscopy images for PDF model based multiple gastrointestinal disease classification. IEEE Access 8:58509–58521

    Google Scholar 

  68. Pozdeev AA, Obukhova NA, Motyko AA (2019) Automatic analysis of endoscopic images for polyps detection and segmentation. IEEE Conf Russ Young Res Electric Electron Eng 2019:1216–1220

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, COMSATS University Islamabad, Wah Campus, Rawalpindi, Pakistan

    Javeria Naz, Muhammad Sharif, Mudassar Raza, Jamal Hussain Shah & Mussarat Yasmin

  2. Faculty of Applied Computing and Technology, Noroff University College, Kristiansand, Norway

    Seifedine Kadry

  3. Department of CSE, Ramco Institute of Technology, Rajapalayam, Tamil Nadu, India

    S. Vimal

Authors
  1. Javeria Naz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Sharif
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mudassar Raza
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jamal Hussain Shah
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mussarat Yasmin
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Seifedine Kadry
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. S. Vimal
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mussarat Yasmin.

Ethics declarations

Conflict of interest

On behalf of all authors, the corresponding author states that there is no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Naz, J., Sharif, M., Raza, M. et al. Recognizing Gastrointestinal Malignancies on WCE and CCE Images by an Ensemble of Deep and Handcrafted Features with Entropy and PCA Based Features Optimization. Neural Process Lett 55, 115–140 (2023). https://doi.org/10.1007/s11063-021-10481-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-021-10481-2

Keywords

Search

Navigation