Skip to main content
SpringerLink
Log in

A monocular SLAM system based on SIFT features for gastroscope tracking

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

During flexible gastroscopy, physicians have extreme difficulties to self-localize. Camera tracking method such as simultaneous localization and mapping (SLAM) has become a research hotspot in recent years, allowing tracking of the endoscope. However, most of the existing solutions have focused on tasks in which sufficient texture information is available, such as laparoscope tracking, and cannot be applied to gastroscope tracking since gastroscopic images have fewer textures than laparoscopic images. This paper proposes a new monocular SLAM framework based on scale-invariant feature transform (SIFT) and narrow-band imaging (NBI), which extracts SIFT features instead of oriented features from accelerated segment test (FAST) and rotated binary robust independent elementary features (BRIEF) features from gastroscopic NBI images, and performs feature retention based on the response sorting strategy for achieving more matches. Experimental results show that the root mean squared error of the proposed algorithm can reach a minimum of 2.074 mm, and the pose accuracy can be improved by up to 25.73% compared with oriented FAST and rotated BRIEF (ORB)-SLAM. SIFT features and response sorting strategy can achieve more accurate matching in gastroscopic NBI images than other features and homogenization strategy, and the proposed algorithm can also run successfully on real clinical gastroscopic data. The proposed algorithm has the potential clinical value to assist physicians in locating the gastroscope during gastroscopy.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Sung H, Ferlay J, Siegel RL, Laversanne M, Soerjomataram I, Jemal A, Bray F (2021) Global Cancer Statistics 2020: GLOBOCAN Estimates of Incidence and Mortality Worldwide for 36 Cancers in 185 Countries. CA: A Cancer J Clin 71:209–249

    Google Scholar 

  2. Katai H, Ishikawa T, Akazawa K, Isobe Y, Miyashiro I, Oda I, Tsujitani S, Ono H, Tanabe S, Fukagawa T, Nunobe S, Kakeji Y, Nashimoto A (2018) Five-year survival analysis of surgically resected gastric cancer cases in Japan: a retrospective analysis of more than 100,000 patients from the nationwide registry of the Japanese Gastric Cancer Association (2001–2007). Gastric Cancer 21:144–154

    Article  Google Scholar 

  3. Fujishiro M, Yoshida S, Matsuda R, Narita A, Yamashita H, Seto Y (2017) Updated evidence on endoscopic resection of early gastric cancer from Japan. Gastric Cancer 20:39–44

    Article  Google Scholar 

  4. Mountney P, Stoyanov D, Davison A, Yang GZ (2006) Simultaneous stereoscope localization and soft-tissue mapping for minimal invasive surgery. Med Image Comput Comput Assist Interv 9:347–54

    Google Scholar 

  5. Mahmoud N, Collins T, Hostettler A, Soler L, Doignon C, Montiel JMM (2019) Live tracking and dense reconstruction for handheld monocular endoscopy. IEEE Trans Med Imaging 38:79–89

    Article  Google Scholar 

  6. Mur-Artal R, Montiel JMM, Tardos JD (2015) ORB-SLAM: a versatile and accurate monocular SLAM system. IEEE Trans Rob 31:1147–1163

    Article  Google Scholar 

  7. Burschka D, Li M, Ishii M, Taylor RH, Hager GD (2005) Scale-invariant registration of monocular endoscopic images to CT-scans for sinus surgery. Med Image Anal 9:413–426

    Article  Google Scholar 

  8. Mountney P, Giannarou S, Elson D, Yang GZ (2009) Optical biopsy mapping for minimally invasive cancer screening. Med Image Comput Comput Assist Interv 12:483–90

    Google Scholar 

  9. Mountney P, Yang GZ (2010) Motion compensated SLAM for image guided surgery. Med Image Comput Comput Assist Interv 13:496–504

    Google Scholar 

  10. Grasa G, Civera J, Guemes A, Munoz V, Montiel JMM (2009) EKF monocular SLAM 3D modeling, measuring and augmented reality from endoscope image sequences. In: 5th Workshop on Augmented Environments for Medical Imaging Including Augmented Reality in Computer Aided Surgery (MICCAI) 2:102–109

  11. Grasa G, Civera J, Montiel JMM (2011) “EKF monocular SLAM with relocalization for laparoscopic sequences,” In: Proceedings of the IEEE international conference on robotics and automation (ICRA), pp 4816–4821

  12. Grasa G, Bernal E, Casado S, Gil I, Montiel JMM (2014) Visual SLAM for handheld monocular endoscope. IEEE Trans Med Imaging 33:135–146

    Article  Google Scholar 

  13. Rublee E, Rabaud V, Konolige K, Bradski G (2011) “ORB: an efficient alternative to SIFT or SURF,” In: Proceedings of the IEEE International Conference on Computer Vision, pp 2564–2571

  14. Mahmoud N, Cirauqui I, Hostettler A, Doignon C, Soler L, Marescaux J, Montiel JMM (2017) “ORBSLAM-based endoscope tracking and 3D reconstruction.” International Workshop on Computer-Assisted and Robotic Endoscopy (CARE), pp. 72–83

  15. Wang C, Oda M, Hayashi Y, Kitasaka T, Honma H, Takabatake H, Mori M, Natori H, Mori K, Fei B, Linte CA (2019) “Visual SLAM for bronchoscope tracking and bronchus reconstruction in bronchoscopic navigation,” In: SPIE Medical Imaging 2019: Image-Guided Procedures, Robotic Interventions, and Modeling, 10951, 109510A

  16. Lamarca J, Parashar S, Bartoli A, Montiel JMM (2021) DefSLAM: tracking and mapping of deforming scenes from monocular sequences. IEEE Trans Rob 37:291–303

    Article  Google Scholar 

  17. Widya R, Monno Y, Imahori K, Okutomi M, Suzuki S, Gotoda T, Miki K (2019) 3D reconstruction of whole stomach from endoscope video using Structure-from-Motion. Annu Int Conf IEEE Eng Med Biol Soc 2019:3900–3904

    Google Scholar 

  18. Widya R, Monno Y, Okutomi M, Suzuki S, Gotoda T, Miki K (2019) Whole stomach 3D reconstruction and frame localization from monocular endoscope video. IEEE J Transl Eng Health Med 7:1–10

    Article  Google Scholar 

  19. Gono K (2015) Narrow band imaging: technology basis and research and development history. Clin Endosc 48:476–480

    Article  Google Scholar 

  20. Yoshida N, Doyama H, Yano T, Horimatsu T, Uedo N, Yamamoto Y, Kakushima N, Kanzaki H, Hori S, Yao K, Oda I, Katada C, Yokoi C, Ohata K, Yoshimura K, Ishikawa H, Muto M (2020) Early gastric cancer detection in high-risk patients: a multicentre randomised controlled trial on the effect of second-generation narrow band imaging. Gut 70:67–75

    Article  Google Scholar 

  21. Zhang Q, Wang F, Chen Z, Wang Z, Zhi F, Liu S, Bai Y (2016) Comparison of the diagnostic efficacy of white light endoscopy and magnifying endoscopy with narrow band imaging for early gastric cancer: a meta-analysis. Gastric Cancer 19:543–552

    Article  Google Scholar 

  22. Lowe DG (2004) Distinctive image features from scale-invariant keypoints. Int J Comput Vision 60:91–110

    Article  Google Scholar 

  23. Liu Y, Liu S, Wang Z (2015) Multi-focus image fusion with dense SIFT. Inf fusion 23:139–155

    Article  Google Scholar 

  24. Zhang T, Zheng W, Cui Z, Zong Y, Yan J, Yan K (2016) A deep neural network-driven feature learning method for multi-view facial expression recognition. IEEE trans multimed 18:2528–2536

    Article  Google Scholar 

  25. Chen S, Zhong S, Xue B, Li X, Zhao L, Chang C (2021) iterative scale-invariant feature transform for remote sensing image registration. IEEE trans geosci remote sens 59:3244–3265

    Article  Google Scholar 

  26. Sturm J, Engelhard N, Endres F, Burgard W, Cremers D (2012) A benchmark for the evaluation of RGB-D SLAM systems. In: 25th IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 573–580

  27. Acharya KA, VenkateshBabu R, Vadhiyar SS (2018) A real-time implementation of SIFT using GPU. J real-time image proc 14:267–277

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Key R&D Program of China under Grant 2019YFB1311501 and in part by the National Natural Science Foundation of China under Grant 62133010.

Author information

Authors and Affiliations

  1. Key Laboratory of Mechanism Theory and Equipment Design of Ministry of Education, Tianjin University, Tianjin, China

    Yifan Wang, Lun Gong & Siyang Zuo

  2. Faculty of Engineering and Information Technology, Robotics Institute, University of Technology Sydney, Sydney, Australia

    Liang Zhao

  3. Tianjin Medical University General Hospital, Tianjin, China

    Xin Chen

Authors
  1. Yifan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Liang Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lun Gong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xin Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Siyang Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Siyang Zuo.

Ethics declarations

Ethics approval

This study was approved by the Ethical Committee of Tianjin Medical University General Hospital, and the approval ID is IRB2022-YX-046–01.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Conflict of interest

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Y., Zhao, L., Gong, L. et al. A monocular SLAM system based on SIFT features for gastroscope tracking. Med Biol Eng Comput 61, 511–523 (2023). https://doi.org/10.1007/s11517-022-02739-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-022-02739-1

Keywords

Search

Navigation