Skip to main content
SpringerLink
Log in

A combined approach of non-subsampled contourlet transform and convolutional neural network to detect gastrointestinal polyp

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The abnormal growth of tissues that disarray the typical organization of cells is popularly known as polyps. The polyp on the gastrointestinal is a primary sign of gastrointestinal cancer. False diagnosis is extremely high using traditional diagnosis procedures that make the polyp diagnosis is a crucial task in real-time colonoscopy. We have developed a polyp detection methodology using a combination of hand-crafted and automated feature extraction techniques. In this study, we have experimented with different convolutional neural network (CNN) architectures and hand-crafted feature extractors to select the best combination. The combined approach of the fine-tuned Xception model with non-subsampled contourlet transform (NSCT) performed significantly well. Besides, we have applied the multi-criteria frame selection technique for selecting the best images from colonoscopy videos. Afterward, the feature extractors have worked on enhanced patch images of selected frames. This study has also experimented with dimensionality reduction techniques to remove irrelevant features from the combined feature vector. We designed an algorithm to localize the polyp regions using the outcomes of patch images. The method did significantly well on several available public datasets. This work might be helpful for the endoscopist during real-time endoscopy to detect polyps.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

References

  1. Bernal J, Tajkbaksh N, Sanchez FJ, Matuszewski BJ, Chen H, Yu L, Angermann Q, Romain O, Rustad B, Balasingham I, Pogorelov K, Choi S, Debard Q, Maier-Hein L, Speidel S, Stoyanov D, Brandao P, Cordova H, Sanchez-Montes C, … Histace A (2017) Comparative validation of polyp detection methods in video colonoscopy: results from the MICCAI 2015 endoscopic vision challenge. IEEE Trans Med Imaging 36:1231–1249. https://doi.org/10.1109/TMI.2017.2664042

    Article  Google Scholar 

  2. Bhatnagar G, Wu QMJ, Liu Z (2013) Directive contrast based multimodal medical image fusion in NSCT domain. IEEE Trans Multimed 15:1014–1024. https://doi.org/10.1109/TMM.2013.2244870

    Article  Google Scholar 

  3. Billah M, Waheed S (2020) Minimum redundancy maximum relevance (mRMR) based feature selection from endoscopic images for automatic gastrointestinal polyp detection. Multimed Tools Appl 79:23633–23643. https://doi.org/10.1007/s11042-020-09151-7

    Article  Google Scholar 

  4. Billah M, Waheed S, Rahman MM (2017) An automatic gastrointestinal polyp detection system in video endoscopy using fusion of color wavelet and convolutional neural network features. Int J Biomed Imaging 2017:1–9. https://doi.org/10.1155/2017/9545920

    Article  Google Scholar 

  5. Brandao P, Mazomenos E, Ciuti G et al (2017) Fully convolutional neural networks for polyp segmentation in colonoscopy. In: medical imaging 2017: computer-aided diagnosis

  6. Breiman L (2001) Random forests. Mach Learn. https://doi.org/10.1023/A:1010933404324

  7. Chowdhary CL, Mittal M, Kumaresan P et al (2020) An efficient segmentation and classification system in medical images using intuitionist possibilistic fuzzy C-mean clustering and fuzzy SVM algorithm. Sensors (Switzerland). https://doi.org/10.3390/s20143903

  8. da Cunha AL, Zhou J, Do MN (2006) The nonsubsampled contourlet transform: theory, design, and applications. IEEE Trans Image Process 15:3089–3101. https://doi.org/10.1109/TIP.2006.877507

    Article  Google Scholar 

  9. Das TK, Lal Chowdhary C, Gao XZ (2020) Chest X-ray investigation: a convolutional neural network approach. J Biomimetics, Biomater Biomed Eng 45:57–70. https://doi.org/10.4028/www.scientific.net/jbbbe.45.57

    Article  Google Scholar 

  10. Deeba F, Bui FM, Wahid KA (2020) Computer-aided polyp detection based on image enhancement and saliency-based selection. Biomed Signal Process Control 55:101530. https://doi.org/10.1016/j.bspc.2019.04.007

    Article  Google Scholar 

  11. Gopalakrishnan K, Khaitan SK, Choudhary A, Agrawal A (2017) Deep convolutional neural networks with transfer learning for computer vision-based data-driven pavement distress detection. Constr Build Mater 157:322–330. https://doi.org/10.1016/j.conbuildmat.2017.09.110

    Article  Google Scholar 

  12. Haralick RM, Dinstein I, Shanmugam K (1973) Textural features for image classification. IEEE Trans Syst Man Cybern SMC-3:610–621. https://doi.org/10.1109/TSMC.1973.4309314

    Article  Google Scholar 

  13. Hasan MM, Islam N, Rahman MM (2020) Gastrointestinal polyp detection through a fusion of contourlet transform and neural features. J King Saud Univ Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.12.013

  14. Hu Y, Liang Z, Song B, Han H, Pickhardt PJ, Zhu W, Duan C, Zhang H, Barish MA, Lascarides CE (2016) Texture feature extraction and analysis for polyp differentiation via computed tomography colonography. IEEE Trans Med Imaging 35:1522–1531. https://doi.org/10.1109/TMI.2016.2518958

    Article  Google Scholar 

  15. Kang J, Gwak J (2019) Ensemble of instance segmentation models for polyp segmentation in colonoscopy images. IEEE Access 7:26440–26447. https://doi.org/10.1109/ACCESS.2019.2900672

    Article  Google Scholar 

  16. Lee JY, Jeong J, Song EM, Ha C, Lee HJ, Koo JE, Yang DH, Kim N, Byeon JS (2020) Real-time detection of colon polyps during colonoscopy using deep learning: systematic validation with four independent datasets. Sci Rep 10:8379. https://doi.org/10.1038/s41598-020-65387-1

    Article  Google Scholar 

  17. Ma C, Ma C, Teriaky A et al (2019) Morbidity and mortality after surgery for nonmalignant colorectal polyps: a 10-year nationwide analysis. Am J Gastroenterol. https://doi.org/10.14309/ajg.0000000000000407

  18. Mamonov AV, Figueiredo IN, Figueiredo PN, Richard Tsai YH (2014) Automated polyp detection in colon capsule endoscopy. IEEE Trans Med Imaging 33:1488–1502. https://doi.org/10.1109/TMI.2014.2314959

    Article  Google Scholar 

  19. Mesejo P, Pizarro D, Abergel A, Rouquette O, Beorchia S, Poincloux L, Bartoli A (2016) Computer-aided classification of gastrointestinal lesions in regular colonoscopy. IEEE Trans Med Imaging 35:2051–2063. https://doi.org/10.1109/TMI.2016.2547947

    Article  Google Scholar 

  20. Ning X, Duan P, Li W, Zhang S (2020) Real-time 3D face alignment using an encoder-decoder network with an efficient deconvolution layer. IEEE Signal Process Lett 27:1944–1948. https://doi.org/10.1109/LSP.2020.3032277

    Article  Google Scholar 

  21. Ning X, Gong K, Li W et al (2020) Feature refinement and filter network for person re-identification. IEEE Trans Circuits Syst Video Technol. https://doi.org/10.1109/TCSVT.2020.3043026

  22. Patino-Barrientos S, Sierra-Sosa D, Garcia-Zapirain B, Castillo-Olea C, Elmaghraby A (2020) Kudo’s classification for colon polyps assessment using a deep learning approach. Appl Sci. https://doi.org/10.3390/app10020501

  23. Peng H, Long F, Ding C (2005) Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27:1226–1238. https://doi.org/10.1109/TPAMI.2005.159

    Article  Google Scholar 

  24. Qadir HA, Shin Y, Solhusvik J, Bergsland J, Aabakken L, Balasingham I (2021) Toward real-time polyp detection using fully CNNs for 2D Gaussian shapes prediction. Med Image Anal 68:101897. https://doi.org/10.1016/j.media.2020.101897

    Article  Google Scholar 

  25. Shin Y, Qadir HA, Aabakken L, Bergsland J, Balasingham I (2018) Automatic colon polyp detection using region based deep CNN and post learning approaches. IEEE Access. 6:40950–40962. https://doi.org/10.1109/ACCESS.2018.2856402

    Article  Google Scholar 

  26. Silva J, Histace A, Romain O, Dray X, Granado B (2014) Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int J Comput Assist Radiol Surg 9:283–293. https://doi.org/10.1007/s11548-013-0926-3

    Article  Google Scholar 

  27. Song M, Emilsson L, Bozorg SR, Nguyen LH, Joshi AD, Staller K, Nayor J, Chan AT, Ludvigsson JF (2020) Risk of colorectal cancer incidence and mortality after polypectomy: a Swedish record-linkage study. Lancet Gastroenterol Hepatol 5:537–547. https://doi.org/10.1016/S2468-1253(20)30009-1

    Article  Google Scholar 

  28. Sornapudi S, Meng F, Yi S (2019) Region-based automated localization of colonoscopy and wireless capsule endoscopy polyps. Appl Sci 9:2404. https://doi.org/10.3390/app9122404

    Article  Google Scholar 

  29. Sornapudi S, Meng F, Yi S (2019) Region-based automated localization of colonoscopy and wireless capsule endoscopy polyps. Appl Sci. https://doi.org/10.3390/app9122404

  30. Tajbakhsh N, Gurudu SR, Liang J (2016) Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging 35:630–644. https://doi.org/10.1109/TMI.2015.2487997

    Article  Google Scholar 

  31. Tajbakhsh N, Shin JY, Gurudu SR, Hurst RT, Kendall CB, Gotway MB, Liang J (2016) Convolutional neural networks for medical image analysis: full training or fine tuning? IEEE Trans Med Imaging 35:1299–1312. https://doi.org/10.1109/TMI.2016.2535302

    Article  Google Scholar 

  32. Urban G, Tripathi P, Alkayali T, Mittal M, Jalali F, Karnes W, Baldi P (2018) Deep learning localizes and identifies polyps in real time with 96% accuracy in screening colonoscopy. Gastroenterology. 155:1069–1078.e8. https://doi.org/10.1053/j.gastro.2018.06.037

    Article  Google Scholar 

  33. Van Der Maaten LJP, Postma EO, Van Den Herik HJ (2009) Dimensionality reduction: a comparative review. J Mach Learn Res. https://doi.org/10.1080/13506280444000102

  34. Wang P, Berzin TM, Glissen Brown JR, Bharadwaj S, Becq A, Xiao X, Liu P, Li L, Song Y, Zhang D, Li Y, Xu G, Tu M, Liu X (2019) Real-time automatic detection system increases colonoscopic polyp and adenoma detection rates: a prospective randomised controlled study. Gut. 68:1813–1819. https://doi.org/10.1136/gutjnl-2018-317500

    Article  Google Scholar 

  35. Wimmer G, Uhl A, Hafner M (2016) A novel filterbank especially designed for the classification of colonic polyps. In: Proceedings - International Conference on Pattern Recognition

  36. Yu L, Chen H, Dou Q, Qin J, Heng PA (2017) Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Heal Informatics 21:65–75. https://doi.org/10.1109/JBHI.2016.2637004

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Mawlana Bhashani Science and Technology University, Tangail, 1902, Bangladesh

    Md. Mahmodul Hasan, Muhammad Minoar Hossain, Shisir Mia, Mohd. Sultan Ahammad & Mohammad Motiur Rahman

Authors
  1. Md. Mahmodul Hasan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Muhammad Minoar Hossain
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shisir Mia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohd. Sultan Ahammad
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Mohammad Motiur Rahman
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Motiur Rahman.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hasan, M.M., Hossain, M.M., Mia, S. et al. A combined approach of non-subsampled contourlet transform and convolutional neural network to detect gastrointestinal polyp. Multimed Tools Appl 81, 9949–9968 (2022). https://doi.org/10.1007/s11042-022-12250-2

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12250-2

Keywords

Search

Navigation