Hai Vu
ABSTRACT
Gastrointestinal Endoscopy in real-time needs to be done quickly and accurately. During an endoscopic examination, the technical doctor must spend a lot of time classifying anatomical landmarks before proceeding with further work such as lesion detection or abnormal region segmentation. The endoscopist also can ensure the completion of the examination with the help of the anatomical landmarks. In this study, we develop a robust and high-performance neural network classifying ten anatomical landmarks in the Upper Gastrointestinal tract. To this end, the proposed method consists of a Siamese Neural Network (SNN) to learn an efficient (dis)similarity metric of a pair of landmarks. We then propose to use the weight that is learned by SNN to a CNN-based structure for the feature extraction. The proposed CNN-based architecture is a lightweight model of Resnet-18 that is designed to achieve the best performance as well as a low cost of computational time. Finally, we utilize an SVM classifier to automatically classify ten anatomical landmarks from the Upper Gastrointestinal endoscopic images. The dataset used in this study is collected in four lighting modes such as (WLI) White-light Imaging, FICE (Flexible spectral imaging color enhancement), BLI (Blue Light Imaging), and LCI (Linked color imaging). The dataset is labeled by endoscopists with more than five years of experience. To enrich the dataset, we also research and evaluate augmentation techniques as effectively as possible. The augmented data includes a combination of the geometrical transforms and the data generated using CycleGAN deep learning networks. The proposed method works well with four lighting modes without depending on a certain lighting mode. It achieved 96.3% sensitivity, 99.6% specificity, and 99.3% accuracy. As a result of the computational time, we achieved approximately 60 FPS (Frame Per Second) using 20W average power with the deep learning framework Pytorch on the Jetson Xavier AGX.
- Shima Ayyoubi Nezhad, Toktam Khatibi, and Masoudreza Sohrabi. 2022. Proposing Novel Data Analytics Method for Anatomical Landmark Identification from Endoscopic Video Frames. Journal of Healthcare Engineering(2022).Google Scholar
- Yuan-Yen Chang, Pai-Chi Li, Ruey-Feng Chang, Chih-Da Yao, Yang-Yuan Chen, Wen-Yen Chang, and Hsu-Heng Yen. 2022. Deep learning-based endoscopic anatomy classification: an accelerated approach for data preparation and model validation. Surgical Endoscopy 36, 6, 3811–3821. https://doi.org/10.1007/s00464-021-08698-2Google ScholarCross Ref
- R. Hadsell, S. Chopra, and Y. LeCun. 2006. Dimensionality Reduction by Learning an Invariant Mapping. In 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’06), Vol. 2. 1735–1742. https://doi.org/10.1109/CVPR.2006.100Google ScholarDigital Library
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Qi He, Sophia Bano, Omer F. Ahmad, Bo Yang, Xin Chen, Pietro Valdastri, Laurence B. Lovat, Danail Stoyanov, and Siyang Zuo. 2020. Deep learning-based anatomical site classification for upper gastrointestinal endoscopy. International Journal of Computer Assisted Radiology and Surgery 15, 7, 1085–1094. https://doi.org/10.1007/s11548-020-02148-5Google ScholarCross Ref
- Andrew Howard, Mark Sandler, Grace Chu, Liang-Chieh Chen, Bo Chen, Mingxing Tan, Weijun Wang, Yukun Zhu, Ruoming Pang, Vijay Vasudevan, Quoc V. Le, and Hartwig Adam. 2019. Searching for MobileNetV3. arXiv. https://doi.org/10.48550/ARXIV.1905.02244Google Scholar
- Ramprasaath R. Selvaraju, Abhishek Das, Ramakrishna Vedantam, Michael Cogswell, Devi Parikh, and Dhruv Batra. 2016. Grad-CAM: Why did you say that? Visual Explanations from Deep Networks via Gradient-based Localization. CoRR abs/1610.02391(2016).Google Scholar
- Karen Simonyan and Andrew Zisserman. 2014. Very Deep Convolutional Networks for Large-Scale Image Recognition. arXiv. https://doi.org/10.48550/ARXIV.1409.1556Google Scholar
- BS (Pharm) Somnath Pal. 2019. Trends in Medical Visits for Digestive Diseases. https://www.uspharmacist.com/article/trends-in-medical-visits-for-digestive-diseasesGoogle Scholar
- Mingjian Sun, Lingyu Ma, Xiufeng Su, Xiaozhong Gao, Zichao Liu, and Liyong Ma. 2022. Channel separation-based network for the automatic anatomical site recognition using endoscopic images. Biomedical Signal Processing and Control 71, 103167. https://doi.org/10.1016/j.bspc.2021.103167Google ScholarCross Ref
- Christian Szegedy, Wei Liu, Yangqing Jia, Pierre Sermanet, Scott Reed, Dragomir Anguelov, Dumitru Erhan, Vincent Vanhoucke, and Andrew Rabinovich. 2015. Going Deeper With Convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Google ScholarCross Ref
- Hirotoshi Takiyama, Tsuyoshi Ozawa, Soichiro Ishihara, Mitsuhiro Fujishiro, Satoki Shichijo, Shuhei Nomura, Motoi Miura, and Tomohiro Tada. 2018. Automatic anatomical classification of esophagogastroduodenoscopy images using deep convolutional neural networks., 7497 pages. https://doi.org/10.1038/s41598-018-25842-6Google Scholar
- Mingxing Tan and Quoc V. Le. 2021. EfficientNetV2: Smaller Models and Faster Training. arXiv. https://doi.org/10.48550/ARXIV.2104.00298Google Scholar
- YuanPu Zheng, Lauren Hawkins, Jordan Wolff, Olga Goloubeva, and Eric Goldberg. 2012. Detection of Lesions During Capsule Endoscopy: Physician Performance Is Disappointing. Official journal of the American College of Gastroenterology | ACG 107, 4. https://journals.lww.com/ajg/Fulltext/2012/04000/Detection_of_Lesions_During_Capsule_Endoscopy_.13.aspxGoogle Scholar
- Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE international conference on computer vision. 2223–2232.Google ScholarCross Ref
Index Terms
- A robust and high-performance neural network for classifying landmarks in upper gastrointestinal endoscopy images
Recommendations
Distinguishing patients with celiac disease by quantitative analysis of videocapsule endoscopy images
Background: Although videocapsule endoscopy images are helpful in the evaluation of celiac disease, their interpretation is subjective. Quantitative disease markers could assist in determining the extent of villous atrophy and response to treatment. ...
Enteroscopic findings of Celiac Disease and their correlation with mucosal histopathologic changes
Background and aimsSingle Balloon Enteroscopy enables us to examine the small bowel for various diseases. It provides a view of the intestinal mucosa with biopsy capability, which may be helpful in search of a mucosal disease such as Celiac Disease. ...
The role of magnetic assisted capsule endoscopy (MACE) to aid visualisation in the upper GI tract
Examination of the upper gastrointestinal tract by a standard endoscope is often thought as a daunting experience to many who have undertaken or are about to undergo the procedure. The overall perceived size of the gastroscope, unpleasantness of ...
Comments