Abstract
Ulcerative colitis (UC) is a long-term condition that needs clinical attention and can be life-threatening. While Mayo Endoscopic Scoring is widely used to stratify patients at higher risk of developing colorectal cancer, the phenotypic endoscopic features involved in the scoring are highly inconsistent. Thus, devising automated methods is required. However, bias in the labels can also trigger such inconsistency and inaccuracy, which makes the use of fully supervised learning not preferable. We propose to exploit a self-supervised learning paradigm for automated MES grading of endoscopic images in UC. To take full advantage of local and global features, we propose to use Swin Transformers in the MoCo-v3 SSL setting. In addition, we provide a comprehensive benchmarking of other existing SSL methods. Our approach with Swin Transformer with MoCo-v3 provides performance boosts in different data size settings.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Ali, S.: Where do we stand in ai for endoscopic image analysis? deciphering gaps and future directions. npj Digital Med. 5(1), 184 (2022). https://doi.org/10.1038/s41746-022-00733-3
Becker, B.G., et al.: Training and deploying a deep learning model for endoscopic severity grading in ulcerative colitis using multicenter clinical trial data. Therapeutic advances in gastrointestinal endoscopy 14 (2021)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607. PMLR (2020)
Chen, X., Xie, S., He, K.: An empirical study of training self-supervised vision transformers. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9620–9629 (2021). https://doi.org/10.1109/ICCV48922.2021.00950
Chen, X., Fan, H., Girshick, R., He, K.: Improved baselines with momentum contrastive learning. arXiv preprint arXiv:2003.04297 (2020)
Cooney, R.M., Warren, B.F., Altman, D.G., Abreu, M.T., Travis, S.P.: Outcome measurement in clinical trials for ulcerative colitis: towards standardisation. Trials 8(1), June 2007. https://doi.org/10.1186/1745-6215-8-17
Dosovitskiy, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020)
He, K., Fan, H., Wu, Y., Xie, S., Girshick, R.: Momentum contrast for unsupervised visual representation learning. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9729–9738 (2020)
Liu, Z., Lin, Y., Cao, Y., Hu, H., Wei, Y., Zhang, Z., Lin, S., Guo, B.: Swin transformer: hierarchical vision transformer using shifted windows. In: 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 9992–10002 (2021)
Misra, I., Maaten, L.v.d.: Self-supervised learning of pretext-invariant representations. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6707–6717 (2020)
Mokter, M.F., Oh, J., Tavanapong, W., Wong, J., Groen, P.C.d.: Classification of ulcerative colitis severity in colonoscopy videos using vascular pattern detection. In: International Workshop on Machine Learning in Medical Imaging, pp. 552–562. Springer (2020)
van den Oord, A., Li, Y., Vinyals, O.: Representation learning with contrastive predictive coding. ArXiv abs/1807.03748 (2018)
Ozawa, T., Ishihara, S., Fujishiro, M., Saito, H., Kumagai, Y., Shichijo, S., Aoyama, K., Tada, T.: Novel computer-assisted diagnosis system for endoscopic disease activity in patients with ulcerative colitis. Gastrointest. Endosc. 89(2), 416–421 (2019)
Polat, G., Kani, H.T., Ergenc, I., Ozen Alahdab, Y., Temizel, A., Atug, O.: Improving the Computer-Aided Estimation of Ulcerative Colitis Severity According to Mayo Endoscopic Score by Using Regression-Based Deep Learning. Inflammatory Bowel Diseases p. izac226 (2022). https://doi.org/10.1093/ibd/izac226
Schroeder, K.W., Tremaine, W.J., Ilstrup, D.M.: Coated oral 5-aminosalicylic acid therapy for mildly to moderately active ulcerative colitis. N. Engl. J. Med. 317(26), 1625–1629 (1987). https://doi.org/10.1056/NEJM198712243172603
Stidham, R.W., et al.: Performance of a deep learning model vs human reviewers in grading endoscopic disease severity of patients with ulcerative colitis. JAMA Netw. Open 2(5), e193963–e193963 (2019)
Xu, Z., Ali, S., Gupta, S., Leedham, S., East, J.E., Rittscher, J.: Patch-level instance-group discrimination with pretext-invariant learning for colitis scoring. In: Machine Learning in Medical Imaging, pp. 101–110 (2022)
Xu, Z., Rittscher, J., Ali, S.: SSL-CPCD: self-supervised learning with composite pretext-class discrimination for improved generalisability in endoscopic image analysis. arXiv:2306.00197 (2023)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Pyatha, A., Xu, Z., Ali, S. (2023). Vision Transformer-Based Self-supervised Learning for Ulcerative Colitis Grading in Colonoscopy. In: Bhattarai, B., et al. Data Engineering in Medical Imaging. DEMI 2023. Lecture Notes in Computer Science, vol 14314. Springer, Cham. https://doi.org/10.1007/978-3-031-44992-5_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-44992-5_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-44991-8
Online ISBN: 978-3-031-44992-5
eBook Packages: Computer ScienceComputer Science (R0)
