Skip to main content
SpringerLink
Log in

Camera Pose Estimation Based on Feature Extraction and Description for Robotic Gastrointestinal Endoscopy

  • Conference paper
  • First Online:
Intelligent Robotics and Applications (ICIRA 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13015))

Included in the following conference series:

Abstract

The application of robotics in gastrointestinal endoscopy has gained more and more attention over the past decade. The localization and navigation of the robotic gastrointestinal endoscopy is very important in robot-assisted gastrointestinal examination and surgery. The camera pose of the robotic gastrointestinal endoscopy can be estimated directly from the image sequence. However, due to the texture-less nature and strong specular reflections of the digestive tract surface, it is hard to detect enough keypoints to estimate the camera pose when using the traditional handcrafted method. In this paper, we propose an end-to-end CNN-based network to deal with this problem. Our network is trained in a self-supervised manner, and the network plays two roles, a dense feature descriptor and a feature detector simultaneously. The network takes the image sequence as input, and the featured keypoints and their corresponding descriptors as outputs. We demonstrate our algorithm on images captured in stomach phantom. The experimental results show that our method can effectively detect and describe the featured keypoints in challenging conditions.

Supported by National Key R&D Program of China (2019YFB1311503), National Natural Science Foundation of China (62073309, 61773365 and 61811540033) and Shenzhen Sci-ence and Technology Program (JCYJ20210324115606018).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 129.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bernhardt, S., Abi-Nahed, J., Abugharbieh, R.: Robust dense endoscopic stereo reconstruction for minimally invasive surgery. In: Menze, B.H., Langs, G., Lu, L., Montillo, A., Tu, Z., Criminisi, A. (eds.) MCV 2012. LNCS, vol. 7766, pp. 254–262. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-36620-8_25

    Chapter  Google Scholar 

  2. Qian, Y., Bai, T., Li, J., et al.: Magnetic-guided capsule endoscopy in the diagnosis of gastrointestinal diseases in minors. Gastroenterol. Res. Pract. 2, 1–8 (2018)

    Article  Google Scholar 

  3. Ozyoruk, K.B., et al.: EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos: Endo-SfMLearner (2020)

    Google Scholar 

  4. Kundu, J.N., Rahul, M.V., Aditya Ganeshan, R., Babu, V.: Object pose estimation from monocular image using multi-view keypoint correspondence. In: Leal-Taixé, L., Roth, S. (eds.) ECCV 2018. LNCS, vol. 11131, pp. 298–313. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-11015-4_23

    Chapter  Google Scholar 

  5. Ren, S.H.K., Girshick, R.: Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Trans. Pattern Anal. Mach. Intell. 39(6), 1137–1149 (2017)

    Article  Google Scholar 

  6. Pentek, Q., Hein, S., Miernik, A., et al.: Image-based 3D surface approximation of the bladder using structure-from-motion for enhanced cystoscopy based on phantom data. Biomed. Tech. (Berl) 63(4), 461–466 (2018)

    Article  Google Scholar 

  7. Widya, A., Monno, Y., Okutomi, M., Suzuki, S., Gotoda, T., Miki, K.: Whole stomach 3D reconstruction and frame localization from monocular endoscope video. IEEE J. Transl. Eng. Health Med. 7, 1–10 (2019)

    Article  Google Scholar 

  8. Widy, A., Monno, Y., Okutomi, M., et al.: Stomach 3D reconstruction based on virtual chromoendoscopic image generation. In: International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC). IEEE (2020)

    Google Scholar 

  9. Lowe, D.G.: Distinctive image features from scaleinvariant keypoints. Int. J. Comput. Vis. 20(2) (2004)

    Google Scholar 

  10. Bay, H., Ess, A., Tuytelaars, T.: Speeded-up robust features. Comput. Vis. Image Underst. 110(3), 404–417 (2008)

    Article  Google Scholar 

  11. Rublee, E., Rabaud, V., Konolige, K., Bradski, G.: ORB: an efficient alternative to SIFT or SURF. In: IEEE International Conference on Computer Vision. IEEE (2011)

    Google Scholar 

  12. Bergen, T., Wittenberg, T., Munzenmayer, C., et al.: A graph-based approach for local and global panorama imaging in cystoscopy. In: SPIE, vol. 8671, no. 1–7 (2013)

    Google Scholar 

  13. Achrafben-Hamadou, D.C, Soussen, C., et al.: A novel 3D surface construction approach: application to three-dimensional endoscopic data. In: IEEE International Conference on Image Processing (ICIP). IEEE (2010)

    Google Scholar 

  14. Daul, C., et al.: From 2D towards 3D cartography of hollow organs. In: Electrical Engineering Computing Science and Automatic Control (CCE). IEEE (2010)

    Google Scholar 

  15. Trinh, D.H., Daul, C., Blondel, W., et al.: Mosaicing of images with few textures and strong illumination changes: application to gastroscopic scenes. In: International Conference on Image Processing (ICIP), pp. 1263–1267. IEEE (2018)

    Google Scholar 

  16. Phan, T., Trinh, D., Lamarque, D., et al.: Dense optical flow for the reconstruction of weakly textured and structured surfaces: application to endoscopy. In: IEEE International Conference on Image Processing. IEEE (2019)

    Google Scholar 

  17. Han, X., Leung, T., Jia, Y., Sukthankar, R., Berg, A.C.: MatchNet: unifying feature and metric learning for patch-based matching. In: Computer Vision & Pattern Recognition. IEEE (2015)

    Google Scholar 

  18. Balntas, V., Johns, E., Tang, L., Mikolajczyk, K.: PN-Net: conjoined triple deep network for learning local image descriptors (2016)

    Google Scholar 

  19. Tian, Y., Fan, B., Wu, F.: L2-Net: deep learning of discriminative patch descriptor in Euclidean space. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2017)

    Google Scholar 

  20. Dusmanu, M., et al.: D2-Net: a trainable CNN for joint description and detection of local features. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE (2019)

    Google Scholar 

  21. Revaud, J., et al.: R2D2: repeatable and reliable detector and descriptor (2019)

    Google Scholar 

  22. He, K., Lu, Y., Sclaroff, S.: Local descriptors optimized for average precision. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE (2018)

    Google Scholar 

  23. Revaud, J., Weinzaepfel, P., Harchaoui, Z., Schmid, C.: EpicFlow: edge-preserving interpolation of correspondences for optical flow. In: Computer Vision & Pattern Recognition. IEEE (2015)

    Google Scholar 

  24. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004). https://doi.org/10.1023/B:VISI.0000029664.99615.94

    Article  Google Scholar 

  25. Brown, M., Lowe, D.G.: Automatic panoramic image stitching using invariant features. Int. J. Comput. Vis. 74(1), 59–73 (2006)

    Article  Google Scholar 

  26. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27(10), 1615–1630 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Shenzhen Institute of Advanced Technology, Chinese Academy of Sciences, Shenzhen, China

    Yuwei Xu, Zeyang Xia & Jing Xiong

  2. University of Chinese Academy of Sciences, Beijing, China

    Yuwei Xu, Zeyang Xia & Jing Xiong

  3. Department of Gastroenterology, Shenzhen University General Hospital, Shenzhen, China

    Lijuan Feng

Authors
  1. Yuwei Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lijuan Feng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zeyang Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jing Xiong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jing Xiong .

Editor information

Editors and Affiliations

  1. Tsinghua University, Beijing, China

    Xin-Jun Liu

  2. Tsinghua University, Beijing, China

    Zhenguo Nie

  3. Beihang University, Beijing, China

    Jingjun Yu

  4. Tsinghua University, Beijing, China

    Fugui Xie

  5. Shandong University, Shandong, China

    Rui Song

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xu, Y., Feng, L., Xia, Z., Xiong, J. (2021). Camera Pose Estimation Based on Feature Extraction and Description for Robotic Gastrointestinal Endoscopy. In: Liu, XJ., Nie, Z., Yu, J., Xie, F., Song, R. (eds) Intelligent Robotics and Applications. ICIRA 2021. Lecture Notes in Computer Science(), vol 13015. Springer, Cham. https://doi.org/10.1007/978-3-030-89134-3_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-89134-3_11

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-89133-6

  • Online ISBN: 978-3-030-89134-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation