Skip to main content
SpringerLink
Log in

Fine-tuned deep neural networks for polyp detection in colonoscopy images

  • Original Paper
  • Published:
Personal and Ubiquitous Computing Aims and scope Submit manuscript

Abstract

Colorectal cancer is one of the leading causes of cancer death worldwide. It can appear in different forms depending on its location, and in most cases, these are tumors called polyps. A diagnostic model which is built can aid doctors but can not replace them. Thus, automated diagnostic technics to detect symptoms of illness or abnormalities in video colonoscopy or wireless capsule endoscopy (WCE) are adopted as an excellent enhancement technique for doctors. In this work, a new computer-assisted diagnosis method for polyp detection is proposed. After a preprocessing step, a fusion of two deep neural networks (DNNs), which were pre-trained on millions of labeled natural images (ImageNet), are fine-tuned and used to extract deep features to perform the polyp detection. The fine-tuning is employed using the Kvasir-seg dataset images. Moreover, the weights of the initial layers of the networks used in this work are frozen. Finally, we concatenated the fully connected outputs of the fine-tuned models to perform the binary classification. The proposed method achieved 0.919, 0.897, and 0.907 on the CVC-ClinicDB dataset, and 0.876, 0.910, and 0.893 on the ETIS-LaribPolypDB dataset in terms of precision, recall, and F-measure metrics, respectively. The results obtained are satisfactory when compared to the state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Bernal J, Sánchez FJ, Fernández-Esparrach G, Gil D, Rodríguez C, Vilariño F (2015) Wm-dova maps for accurate polyp highlighting in colonoscopy: Validation vs. saliency maps from physicians. Computerized Medical Imaging and Graphics 43:99–111

    Article  Google Scholar 

  2. Bernal J, Sánchez J, Vilarino F (2012) Towards automatic polyp detection with a polyp appearance model. Pattern Recogn 45(9):3166–3182

    Article  Google Scholar 

  3. Bernal J, Tajkbaksh N, Sánchez FJ, Matuszewski BJ, Chen H, Yu L, Angermann Q, Romain O, Rustad B, Balasingham I et al (2017) Comparative validation of polyp detection methods in video colonoscopy: results from the miccai 2015 endoscopic vision challenge. IEEE Trans Med Imaging 36 (6):1231–1249

    Article  Google Scholar 

  4. Charfi S, El Ansari M (2018) Computer-aided diagnosis system for colon abnormalities detection in wireless capsule endoscopy images. Multimed Tools Appl 77(3):4047–4064

    Article  Google Scholar 

  5. Deeba F, Bui FM, Wahid KA (2020) Computer-aided polyp detection based on image enhancement and saliency-based selection. Biomed Signal Process Control 55(101):530

    Google Scholar 

  6. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on computer vision and pattern recognition. Ieee, pp 248–255

  7. El Ansari M, Charfi S (2017) Computer-aided system for polyp detection in wireless capsule endoscopy images. In: 2017 International conference on wireless networks and mobile communications (WINCOM). IEEE, pp 1–6

  8. Ellahyani A, El Ansari M, El Jaafari I (2016) Traffic sign detection and recognition based on random forests. Appl Soft Comput 46:805–815

    Article  Google Scholar 

  9. Ellahyani A, El Ansari M, Lahmyed R, Trémeau A (2018) Traffic sign recognition method for intelligent vehicles. JOSA A 35(11):1907–1914

    Article  Google Scholar 

  10. Ellahyani A, El Jaafari I, Charfi S, El Ansari M (2021) Detection of abnormalities in wireless capsule endoscopy based on extreme learning machine. SIViP 15:877–884. https://doi.org/10.1007/s11760-020-01809-x

    Article  Google Scholar 

  11. Gauen K, Rangan R, Mohan A, Lu YH, Liu W, Berg AC (2017) Low-power image recognition challenge. In: 2017 22Nd asia and south pacific design automation conference (ASP-DAC). IEEE, pp 99–104

  12. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H (2017) Mobilenets: Efficient convolutional neural networks for mobile vision applications

  13. Iwahori Y, Shinohara T, Hattori A, Woodham RJ, Fukui S, Bhuyan MK, Kasugai K (2013) Automatic polyp detection in endoscope images using a hessian filter. In: MVA, pp 21–24

  14. Jha D, Smedsrud PH, Riegler MA, Halvorsen P, de Lange T, Johansen D, Johansen HD (2020) Kvasir-seg: A segmented polyp dataset. In: International conference on multimedia modeling. Springer, pp 451–462

  15. Kaminski MF, Regula J, Kraszewska E, Polkowski M, Wojciechowska U, Didkowska J, Zwierko M, Rupinski M, Nowacki MP, Butruk E (2010) Quality indicators for colonoscopy and the risk of interval cancer. N Engl J Med 362(19):1795–1803

    Article  Google Scholar 

  16. Kopelman Y, Gal O, Jacob H, Siersema P, Cohen A et al (2019) Automated polyp detection system in colonoscopy using deep learning and image processing techniques. J Gastroenterol Compl 3(1):101

    Google Scholar 

  17. Li Z, Hoiem D (2017) Learning without forgetting. IEEE Trans Pattern Anal Machine Intell 40(12):2935–2947

    Article  Google Scholar 

  18. Lu S, Lu Z, Zhang YD (2019) Pathological brain detection based on alexnet and transfer learning. J Comput Sci 30:41–47

    Article  Google Scholar 

  19. Ma Y, Chen X, Sun B (2020) Polyp detection in colonoscopy videos by bootstrapping via temporal consistency. In: 2020 IEEE 17Th international symposium on biomedical imaging (ISBI). IEEE, pp 1360–1363

  20. Park S, Lee M, Kwak N (2015) Polyp detection in colonoscopy videos using deeply-learned hierarchical features. Seoul National University

  21. Park SY, Sargent D (2016) Colonoscopic polyp detection using convolutional neural networks. In: Medical imaging 2016: Computer-aided diagnosis, vol 9785. International society for optics and photonics, p 978528

  22. Pogorelov K, Randel KR, Griwodz C, Eskeland SL, de Lange T, Johansen D, Spampinato C, Dang-Nguyen DT, Lux M, Schmidt PT et al (2017) Kvasir: a multi-class image dataset for computer aided gastrointestinal disease detection. In: Proceedings of the 8th ACM on multimedia systems conference, pp 164–169

  23. Pogorelov K, Riegler M, Eskeland SL, de Lange T, Johansen D, Griwodz C, Schmidt PT, Halvorsen P (2017) Efficient disease detection in gastrointestinal videos–global features versus neural networks. Multimed Tools Appl 76(21):22,493–22,525

    Article  Google Scholar 

  24. Qadir HA, Balasingham I, Solhusvik J, Bergsland J, Aabakken L, Shin Y (2019) Improving automatic polyp detection using cnn by exploiting temporal dependency in colonoscopy video. IEEE Journal of Biomedical and Health Informatics

  25. Qadir HA, Shin Y, Solhusvik J, Bergsland J, Aabakken L, Balasingham I (2021) Toward real-time polyp detection using fully cnns for 2d gaussian shapes prediction. Med Image Anal 68(101):897

    Google Scholar 

  26. Ribeiro E, Uhl A, Häfner M (2016) Colonic polyp classification with convolutional neural networks. In: 2016 IEEE 29Th international symposium on computer-based medical systems (CBMS). IEEE, pp 253–258

  27. Riegler M, Pogorelov K, Eskeland SL, Schmidt PT, Albisser Z, Johansen D, Griwodz C, Halvorsen P, Lange TD (2017) From annotation to computer-aided diagnosis: Detailed evaluation of a medical multimedia system. ACM Transactions on Multimedia Computing Communications, and Applications (TOMM) 13(3):1–26

    Article  Google Scholar 

  28. Russakovsky O, Deng J, Su H, Krause J, Satheesh S, Ma S, Huang Z, Karpathy A, Khosla A, Bernstein M et al (2015) Imagenet large scale visual recognition challenge. Int J Comput Vision 115(3):211–252

    Article  MathSciNet  Google Scholar 

  29. Sajjad M, Khan S, Muhammad K, Wu W, Ullah A, Baik SW (2019) Multi-grade brain tumor classification using deep cnn with extensive data augmentation. J Comput Sci 30:174–182

    Article  Google Scholar 

  30. Sermanet P, Eigen D, Zhang X, Mathieu M, Fergus R, LeCun Y (2013) Overfeat: Integrated recognition, localization and detection using convolutional networks. In: The imagenet large scale visual recognition challenge 2013 (ILSVRC2013). arXiv:1312.6229

  31. Ševo I, Avramović A, Balasingham I, Elle OJ, Bergsland J, Aabakken L (2016) Edge density based automatic detection of inflammation in colonoscopy videos. Comput Biol Med 72:138–150

    Article  Google Scholar 

  32. Shin Y, Balasingham I (2017) Comparison of hand-craft feature based svm and cnn based deep learning framework for automatic polyp classification. In: 2017 39Th annual international conference of the IEEE engineering in medicine and biology society (EMBC). IEEE, pp 3277–3280

  33. Silva J, Histace A, Romain O, Dray X, Granado B (2014) Toward embedded detection of polyps in wce images for early diagnosis of colorectal cancer. Int J CARS 9(2):283–293

    Article  Google Scholar 

  34. Simonyan K, Zisserman A (2019) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations, ICLR 2019. arXiv:1409.1556

  35. Sornapudi S, Meng F, Yi S (2019) Region-based automated localization of colonoscopy and wireless capsule endoscopy polyps. Appl Sci 9(12):2404

    Article  Google Scholar 

  36. Tajbakhsh N, Gurudu SR, Liang J (2015) Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans Med Imaging 35(2):630–644

    Article  Google Scholar 

  37. Wienbrandt L, Kässens JC, Hübenthal M, Ellinghaus D (2019) 1000× faster than plink: Combined fpga and gpu accelerators for logistic regression-based detection of epistasis. J Comput Sci 30:183–193

    Article  Google Scholar 

  38. Wittenberg T, Zobel P, Rathke M, Mühldorfer S (2019) Computer aided detection of polyps in whitelight-colonoscopy images using deep neural networks. Current Direct Biomed Eng 5(1):231– 234

    Article  Google Scholar 

  39. Yu L, Chen H, Dou Q, Qin J, Heng PA (2016) Integrating online and offline three-dimensional deep learning for automated polyp detection in colonoscopy videos. IEEE J Biomed Health Inform 21 (1):65–75

    Article  Google Scholar 

  40. Zhu H, Fan Y, Liang Z (2010) Improved curvature estimation for shape analysis in computer-aided detection of colonic polyps. In: International MICCAI workshop on computational challenges and clinical opportunities in virtual colonoscopy and abdominal imaging. Springer, pp 9–14

Download references

Acknowledgements

This work was supported by the Ministry of National Education, Vocational Training, Higher Education and Scientific Research (MNEVTHESR), The Ministry of Industry, Trade and Green and Digital Economy (MITGDE), Digital Development Agency (DDA) and National Center for Scientific and Technical Research (NCSTR). Project number: ALKHAWARIZMI/2020/20

Author information

Authors and Affiliations

  1. LabSIE, Department of Computer Science, Multidisciplinary Faculty, Ibn Zohr University, Ouarzazate, Morocco

    Ayoub Ellahyani & Ilyas El Jaafari

  2. LabSIV, Department of Computer Science, Faculty of Science, Ibn Zohr University, Agadir, Morocco

    Said Charfi

  3. Informatics and Applications Laboratory, Faculty of Sciences, Moulay Ismail University, Meknès, Morocco

    Mohamed El Ansari

  4. LabSIV, Faculty of Science, Ibn Zohr University, Agadir, Morocco

    Mohamed El Ansari

Authors
  1. Ayoub Ellahyani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ilyas El Jaafari
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Said Charfi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mohamed El Ansari
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ayoub Ellahyani.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ellahyani, A., Jaafari, I.E., Charfi, S. et al. Fine-tuned deep neural networks for polyp detection in colonoscopy images. Pers Ubiquit Comput 27, 235–247 (2023). https://doi.org/10.1007/s00779-021-01660-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-021-01660-y

Keywords

Search

Navigation