Skip to main content

Advertisement

SpringerLink
Log in

Automated detection of ulcers and erosions in capsule endoscopy images using a convolutional neural network

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Capsule endoscopy (CE) is an important tool in the management of patients with known or suspected inflammatory bowel disease. Ulcers and erosions of the enteric mucosa are prevalent findings in these patients. They frequently occur together, and their identification in CE is crucial for an accurate evaluation of disease severity. Nevertheless, reviewing CE images is a time-consuming task, and the risk of overlooking lesions is significant.

Over the last decade, artificial intelligence (AI) has emerged as a means for overcoming these pitfalls. Of all AI methods, convolutional neural networks (CNN), due to their complex multilayer architecture present the best results in medical image analysis, particularly capsule endoscopy. Therefore, we aimed to develop a CNN for the automatic identification of ulcers and erosions in the small bowel mucosa. A total of 1483 CE exams (PillCam SB3®) performed at a single center between 2015 and 2020 were analysed. From these exams, a total of 6130 frames of the enteric mucosa were obtained, 4233 containing enteric ulcers and erosions, and the remaining containing normal mucosa or other findings. Ulcers and erosions were stratified according to Saurin’s classification for bleeding potential: P1E—erosions with intermediate bleeding risk; P1U—ulcers with intermediate bleeding risk; P2U—ulcers with high bleeding risk. For automatic identification of these lesions, these images were inserted into a CNN model with transfer learning. The pool of images was divided for constitution of training and validation datasets, comprising 80% and 20% of the total number of images, respectively. The output provided by the CNN was compared to the classification provided by a consensus of specialists. After optimizing the neural architecture of the algorithm, our model was able to automatically detect and distinguish ulcers and erosions (any bleeding potential) in the small intestine mucosa with an accuracy of 95.6%, sensitivity of 90.8%, and a specificity of 97.1%. We believe that our study lays the foundation for the development and application of effective AI tools to CE. These techniques should improve diagnostic accuracy and reading efficiency.

Graphic abstract

Schematic representation of the workflow and summary of the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Abbreviations

CE:

Capsule endoscopy

OGIB:

Obscure gastrointestinal bleeding

CD:

Crohn’s disease

AI:

Artificial intelligence

CNN:

Convolutional neural networks

ROC:

Receiver operating characteristic curves

AUROC:

Area under receiver operating characteristic curves

SD:

Standard deviation

CI:

Confidence intervals

CECDAI:

Capsule Endoscopy Crohn’s Disease Activity Index

References

  1. Teshima CW, Kuipers EJ, van Zanten SV et al (2011) Double balloon enteroscopy and capsule endoscopy for obscure gastrointestinal bleeding: an updated meta-analysis. J Gastroenterol Hepatol 26:796–801

    Article  Google Scholar 

  2. Le Berre C, Trang-Poisson C, Bourreille A (2019) Small bowel capsule endoscopy and treat-to-target in Crohn’s disease: a systematic review. World J Gastroenterol 25:4534–4554

    Article  Google Scholar 

  3. Sturm A, Maaser C, Calabrese E et al (2018) ECCO-ESGAR guideline for diagnostic assessment in IBD Part 2: IBD scores and general principles and technical aspects. J Crohn’s Colitis 13:273–284

    Article  Google Scholar 

  4. Arimoto J, Endo H, Kato T et al (2016) Clinical value of capsule endoscopy for detecting small bowel lesions in patients with intestinal Behçet’s disease. Dig Endosc 28:179–185

    Article  Google Scholar 

  5. Rondonotti E, Pennazio M, Toth E et al (2008) Small-bowel neoplasms in patients undergoing video capsule endoscopy: a multicenter European study. Endoscopy 40:488–495

    Article  CAS  Google Scholar 

  6. Shahidi NC, Ou G, Svarta S et al (2012) Factors associated with positive findings from capsule endoscopy in patients with obscure gastrointestinal bleeding. Clin Gastroenterol Hepatol 10:1381–1385

    Article  Google Scholar 

  7. Goenka MK, Majumder S, Kumar S et al (2011) Single center experience of capsule endoscopy in patients with obscure gastrointestinal bleeding. World J Gastroenterol 17:774–778

    Article  Google Scholar 

  8. Barash Y, Azaria L, Soffer S et al (2021) Ulcer severity grading in video capsule images of patients with Crohn’s disease: an ordinal neural network solution. Gastrointest Endosc 93:187–192

    Article  Google Scholar 

  9. Leenhardt R, Vasseur P, Li C et al (2019) A neural network algorithm for detection of GI angiectasia during small-bowel capsule endoscopy. Gastrointest Endosc 89:189–194

    Article  Google Scholar 

  10. Wang A, Banerjee S, Barth BA et al (2013) Wireless capsule endoscopy. Gastrointest Endosc 78:805–815

    Article  Google Scholar 

  11. Yasaka K, Akai H, Abe O et al (2018) Deep learning with convolutional neural network for differentiation of liver masses at dynamic contrast-enhanced CT: a preliminary study. Radiology 286:887–896

    Article  Google Scholar 

  12. Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118

    Article  CAS  Google Scholar 

  13. Gargeya R, Leng T (2017) Automated identification of diabetic retinopathy using deep learning. Ophthalmology 124:962–969

    Article  Google Scholar 

  14. Klang E, Barash Y, Margalit RY et al (2020) Deep learning algorithms for automated detection of Crohn’s disease ulcers by video capsule endoscopy. Gastrointest Endosc 91:606–13.e2

    Article  Google Scholar 

  15. Aoki T, Yamada A, Aoyama K et al (2019) Automatic detection of erosions and ulcerations in wireless capsule endoscopy images based on a deep convolutional neural network. Gastrointest Endosc 89:357–63.e2

    Article  Google Scholar 

  16. Allez M, Lemann M, Bonnet J et al (2002) Long term outcome of patients with active Crohn’s disease exhibiting extensive and deep ulcerations at colonoscopy. Am J Gastroenterol 97:947–953

    PubMed  Google Scholar 

  17. Rondonotti E, Spada C, Adler S et al (2018) Small-bowel capsule endoscopy and device-assisted enteroscopy for diagnosis and treatment of small-bowel disorders: European Society of Gastrointestinal Endoscopy (ESGE) Technical Review. Endoscopy 50:423–446

    Article  Google Scholar 

  18. Leenhardt R, Buisson A, Bourreille A et al (2020) Nomenclature and semantic descriptions of ulcerative and inflammatory lesions seen in Crohn’s disease in small bowel capsule endoscopy: an international Delphi consensus statement. United Eur Gastroenterol J 8:99–107

    Article  Google Scholar 

  19. Saurin JC, Delvaux M, Gaudin JL et al (2003) Diagnostic value of endoscopic capsule in patients with obscure digestive bleeding: blinded comparison with video push-enteroscopy. Endoscopy 35:576–584

    Article  Google Scholar 

  20. Gal E, Geller A, Fraser G et al (2008) Assessment and validation of the new capsule endoscopy Crohn’s disease activity index (CECDAI). Dig Dis Sci 53:1933–1937

    Article  Google Scholar 

  21. Chollet F (2017) Xception: Deep learning with depthwise separable convolutions.

  22. Deng J, Dong W, Socher R et al (2009) ImageNet: a large-scale hierarchical image database. 2009 IEEE Conference on Computer Vision and Pattern Recognition. pp 248–55

  23. Abadi M, Barham P, Chen J et al (2016) TensorFlow: a system for large-scale machine learning. Proceedings of the 12th USENIX conference on Operating Systems Design and Implementation. Savannah, GA, USA: USENIX Association. pp 265–83

  24. Pedregosa F, Varoquaux G, Gramfort A et al (2011) Scikit-learn: machine learning in python. J Mach Learn Res 12:2825–2830

    Google Scholar 

  25. Gralnek IM, Defranchis R, Seidman E et al (2008) Development of a capsule endoscopy scoring index for small bowel mucosal inflammatory change. Aliment Pharmacol Ther 27:146–154

    Article  CAS  Google Scholar 

  26. Aoki T, Yamada A, Hirata Y et al (2018) Etiology and long-term rebleeding of endoscopic ulcerative lesions in the small bowel in patients with obscure gastrointestinal bleeding: a multicenter cohort study. J Gastroenterol Hepatol 33:1327–1334

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Gastroenterology, São João University Hospital, Alameda Professor Hernâni Monteiro, 4200-427, Porto, Portugal

    João Afonso, Miguel Mascarenhas Saraiva, Hélder Cardoso, Tiago Ribeiro, Patrícia Andrade & Guilherme Macedo

  2. WGO Gastroenterology and Hepatology Training Center, Porto, Portugal

    João Afonso, Miguel Mascarenhas Saraiva, Hélder Cardoso, Tiago Ribeiro, Patrícia Andrade & Guilherme Macedo

  3. Faculty of Medicine of the University of Porto, Alameda Professor Hernâni Monteiro, 4200-427, Porto, Portugal

    Miguel Mascarenhas Saraiva, Hélder Cardoso, Patrícia Andrade & Guilherme Macedo

  4. Faculty of Engineering of the University of Porto, Rua Dr. Roberto Frias, 4200-465, Porto, Portugal

    J. P. S. Ferreira, Marco Parente & Renato N. Jorge

  5. INEGI - Institute of Science and Innovation in Mechanical and Industrial Engineering, Rua Dr. Roberto Frias, 4200-465, Porto, Portugal

    J. P. S. Ferreira, Marco Parente & Renato N. Jorge

Authors
  1. João Afonso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Miguel Mascarenhas Saraiva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. P. S. Ferreira
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hélder Cardoso
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Tiago Ribeiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Patrícia Andrade
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Marco Parente
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Renato N. Jorge
    View author publications

    You can also search for this author in PubMed Google Scholar

  9. Guilherme Macedo
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

João Afonso and Miguel Mascarenhas Saraiva: study design, revision of CE videos, image extraction, construction, and development of the CNN; drafting of the manuscript; critical revision of the manuscript.

João P.S. Ferreira: construction and development of the CNN, statistical analysis, critical revision of the manuscript.

Hélder Cardoso: revision of CE videos, image extraction; critical revision of the manuscript.

Tiago Ribeiro: bibliographic review, drafting of the manuscript, critical revision of the manuscript.

Patrícia Andrade, Marco L.P. Parente, Renato N. Jorge, Guilherme Macedo: study design, critical revision of the manuscript.

All authors approved the final version of the manuscript.

Corresponding author

Correspondence to João Afonso.

Ethics declarations

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Afonso, J., Saraiva, M.M., Ferreira, J.P.S. et al. Automated detection of ulcers and erosions in capsule endoscopy images using a convolutional neural network. Med Biol Eng Comput 60, 719–725 (2022). https://doi.org/10.1007/s11517-021-02486-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-021-02486-9

Keywords

Search

Navigation