Abstract
Endoscopic measurement of ulcerative colitis (UC) severity is important since endoscopic disease severity may better predict future outcomes in UC than symptoms. However, it is difficult to evaluate the endoscopic severity of UC objectively because of the non-uniform nature of endoscopic features associated with UC, and large variations in their patterns. In this paper, we propose a method to classify UC severity in colonoscopy videos by learning from confusion. The similar looking frames from the colonoscopy videos generate similar features, and the Convolutional Neural Network (CNN) model trained using these similar features is confused. Therefore, it cannot provide accurate classification. By isolating these similar frames (features), we potentially reduce model confusion. We propose a new training strategy to isolate these similar frames (features), and a CNN based method for classifying UC severity in colonoscopy videos using the new training strategy. The experiments show that the proposed method for classifying UC severity increases classification effectiveness significantly.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
U.S. National Library of Medicine. Ulcerative colitis. https://ghr.nlm.nih.gov/condition/ulcerative-colitis. Accessed June 2022
Xie, T., et al.: Ulcerative colitis endoscopic index of severity (UCEIS) versus mayo endoscopic score (MES) in guiding the need for colectomy in patients with acute severe colitis. Gastroenterol. Rep. 6(1), 38–44 (2018)
Kappelman, M.D., Rifas-Shiman, S.L., Kleinman, K., et al.: The prevalence and geographic distribution of Crohn’s disease and ulcerative colitis in the United States. Clin. Gastroenterol. Hepatol. 5(12), 1424–1429 (2007)
Rutter, M., Saunders, B., et al.: Severity of inflammation is a risk factor for colorectal neoplasia in ulcerative colitis. Gastroenterology 126(2), 451–459 (2004)
Nosato, H., Sakanashi, H., Takahashi, E., Murakawa, M.: An objective evaluation method of ulcerative colitis with optical colonoscopy images based on higher order local auto-correlation features. In: 2014 IEEE 11th International Symposium on Biomedical Imaging (ISBI), pp. 89–92. IEEE (2014)
Alammari, A., Islam, A.R., Oh, J., Tavanapong, W., Wong, J., De Groen, P. C.: Classification of ulcerative colitis severity in colonoscopy videos using CNN. In: Proceedings of the 9th International Conference on Information Management and Engineering, Barcelona, Spain, pp. 139–144 (2017)
Tejaswini, S.V.L.L., Mittal, B., Oh, J., Tavanapong, W., Wong, J., de Groen, P.C.: Enhanced approach for classification of ulcerative colitis severity in colonoscopy videos using CNN. In: Bebis, G., et al. (eds.) ISVC 2019. LNCS, vol. 11845, pp. 25–37. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-33723-0_3
Mokter, M.F., Oh, J., Tavanapong, W., Wong, J., de Groen, P.C.: Classification of ulcerative colitis severity in colonoscopy videos using vascular pattern detection. In: Liu, M., Yan, P., Lian, C., Cao, X. (eds.) MLMI 2020. LNCS, vol. 12436, pp. 552–562. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59861-7_56
Takenaka, K., et al.: Development and validation of a deep neural network for accurate evaluation of endoscopic images from patients with ulcerative colitis. Gastroenterology 158(8), 2150–2157 (2020)
Bhambhvani, H.P., Zamora, A.: Deep learning enabled classification of Mayo endoscopic subscore in patients with ulcerative colitis. Eur. J. Gastroenterol. Hepatol. 33(5), 645–649 (2021)
Luo, X., Zhang, J., Li, Z., Yang, R.: Diagnosis of ulcerative colitis from endoscopic images based on deep learning. Biomed. Signal Process. Control 73, 103443 (2022)
Schwab, E., et al.: Automatic estimation of ulcerative colitis severity from endoscopy videos using ordinal multi-instance learning. Comput. Methods Biomech. Biomed. Eng. Imaging Vis. 10(4), 425–433 (2022)
Harada, S., Bise, R., Hayashi, H., Tanaka, K., Uchida, S.: Order-guided disentangled representation learning for ulcerative colitis classification with limited labels. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12902, pp. 471–480. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87196-3_44
Sutton, R.T., Zai͏̈ane, O.R., Goebel, R., Baumgart, D.C.: Artificial intelligence enabled automated diagnosis and grading of ulcerative colitis endoscopy images. Sci. Rep. 12(1), 1–10 (2022)
Borgli, H., et al.: HyperKvasir, a comprehensive multi-class image and video dataset for gastrointestinal endoscopy. Sci. Data 7(1), 1–14 (2020)
Wang, C., et al.: Pulmonary image classification based on inception-v3 transfer learning model. IEEE Access 7, 146533–146541 (2019)
Sarwinda, D., Paradisa, R.H., Bustamam, A., Anggia, P.: Deep learning in image classification using residual network (ResNet) variants for detection of colorectal cancer. Procedia Comput. Sci. 179, 423–431 (2021)
Mascarenhas, S., Agarwal, M.: A comparison between VGG16, VGG19 and ResNet50 architecture frameworks for Image Classification. In: 2021 International Conference on Disruptive Technologies for Multi-Disciplinary Research and Applications (CENTCON), vol. 1, pp. 96–99. IEEE, November 2021
Dubey, N., Bhagat, E., Rana, S., Pathak, K.: A novel approach to detect plant disease using DenseNet-121 neural network. In: Zhang, Y.D., Senjyu, T., So-In, C., Joshi, A. (eds.) Smart Trends in Computing and Communications. LNNS, vol. 396, pp. 63–74. Springer, Singapore (2023). https://doi.org/10.1007/978-981-16-9967-2_7
Liao, L., Li, H., Shang, W., Ma, L.: An empirical study of the impact of hyperparameter tuning and model optimization on the performance properties of deep neural networks. ACM Trans. Softw. Eng. Methodol. (TOSEM) 31(3), 1–40 (2022)
Liu, R., Krishnan, S., Elmore, A.J., Franklin, M.J.: Understanding and optimizing packed neural network training for hyper-parameter tuning. In: Proceedings of the Fifth Workshop on Data Management for End-To-End Machine Learning, pp. 1–11 (2021)
Salahuddin, Z., Woodruff, H.C., Chatterjee, A., Lambin, P.: Transparency of deep neural networks for medical image analysis: a review of interpretability methods. Comput. Biol. Med. 140, 105111 (2022)
Cheng, J., et al.: ResGANet: residual group attention network for medical image classification and segmentation. Med. Image Anal. 76, 102313 (2022)
Xu, J., Pan, Y., Pan, X., Hoi, S., Yi, Z., Xu, Z.: RegNet: self-regulated network for image classification. IEEE Trans. Neural Netw. Learn. Syst. (2022)
Mao, J., et al.: Pseudo-labeling generative adversarial networks for medical image classification. Comput. Biol. Med. 147, 105729 (2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
Tavanapong and Oh have equity interest and management roles in EndoMetric Corp. Dr. de Groen serves on the Scientific Advisory Board of EndoMetric Corp. This work is partially supported by the NIH Grant No. 1R01DK106130-01A1. Findings, opinions, and conclusions expressed in this paper do not necessarily reflect the view of the funding agency.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Mokter, M.F., Idris, A., Oh, J., Tavanapong, W., de Groen, P.C. (2022). Severity Classification of Ulcerative Colitis in Colonoscopy Videos by Learning from Confusion. In: Bebis, G., et al. Advances in Visual Computing. ISVC 2022. Lecture Notes in Computer Science, vol 13598. Springer, Cham. https://doi.org/10.1007/978-3-031-20713-6_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-20713-6_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-20712-9
Online ISBN: 978-3-031-20713-6
eBook Packages: Computer ScienceComputer Science (R0)