Abstract
For monocular endoscope motion estimation, traditional algorithms often suffer from poor robustness when encountering uninformative or dark frames since they only use prominent image features. In contrast, deep learning methods based on an end-to-end framework have achieved promising performance by estimating the 6-DOF pose directly. However, the existing techniques overly depend on the mass high-precision labelled 6-DOF pose data, which is difficult to obtain in practical scenarios. In this work, we propose a fast yet robust method for monocular endoscope motion estimation named Deep Motion Flow Estimation (DMFE). Specifically, we propose an innovative Key Points Encoder (KPE) supervised by Speeded-up Robust Features (SURF) flow to extract the salient features of endoscopic images. Aiming to ensure real-time capability, we propose a novel 3D motion transfer algorithm to reduce the computational complexity of the essential matrix. Extensive experiments on clinical and virtual colon datasets demonstrate the superiority of our method against the traditional methods, which can provide visual navigation assistance for doctors or robotic endoscopes in real-world scenarios.
Supported by National Natural Science Foundation of China (62073309), Guangdong Basic and Applied Basic Research Foundation (2022B1515020042) and Shenzhen Science and Technology Program (JCYJ20210324115606018).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Armin, M.A., Barnes, N., Alvarez, J., Li, H., Grimpen, F., Salvado, O.: Learning camera pose from optical colonoscopy frames through deep convolutional neural network (CNN). In: Cardoso, M.J., et al. (eds.) CARE/CLIP -2017. LNCS, vol. 10550, pp. 50–59. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67543-5_5
Bay, H., Tuytelaars, T., Van Gool, L.: SURF: speeded up robust features. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3951, pp. 404–417. Springer, Heidelberg (2006). https://doi.org/10.1007/11744023_32
Bell, C.S., Obstein, K.L., Valdastri, P.: Image partitioning and illumination in image-based pose detection for teleoperated flexible endoscopes. Artif. Intell. Med. 59(3), 185–196 (2013)
Chen, G., Pham, M.T., Redarce, T.: Sensor-based guidance control of a continuum robot for a semi-autonomous colonoscopy. Robot. Auton. Syst. 57(6–7), 712–722 (2009)
Chen, H., et al.: Participation and yield of a population-based colorectal cancer screening programme in China. Gut 68(8), 1450–1457 (2019). https://doi.org/10.1136/gutjnl-2018-317124
Fischler, M.A., Bolles, R.C.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)
Givens, C.R., Shortt, R.M.: A class of Wasserstein metrics for probability distributions. Mich. Math. J. 31(2), 231–240 (1984)
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)
Jiang, W., Zhou, Y., Wang, C., Peng, L., Yang, Y., Liu, H.: Navigation strategy for robotic soft endoscope intervention. Int. J. Med. Robot. Comput. Assist. Surg. 16(2), e2056 (2020)
Liu, Z., et al.: Swin transformer: hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 10012–10022 (2021)
Muja, M., Lowe, D.G.: Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2227–2240 (2014)
Ozyoruk, K.B., et al.: EndoSLAM dataset and an unsupervised monocular visual odometry and depth estimation approach for endoscopic videos. Med. Image Anal. 71, 102058 (2021)
Puerto-Souza, G.A., Staranowicz, A.N., Bell, C.S., Valdastri, P., Mariottini, G.-L.: A comparative study of ego-motion estimation algorithms for teleoperated robotic endoscopes. In: Luo, X., Reichl, T., Mirota, D., Soper, T. (eds.) CARE 2014. LNCS, vol. 8899, pp. 64–76. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13410-9_7
Recasens, D., Lamarca, J., Fácil, J.M., Montiel, J., Civera, J.: Endo-depth-and-motion: reconstruction and tracking in endoscopic videos using depth networks and photometric constraints. IEEE Robot. Autom. Lett. 6(4), 7225–7232 (2021)
Sevilla-Lara, L., Sun, D., Learned-Miller, E.G., Black, M.J.: Optical flow estimation with channel constancy. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8689, pp. 423–438. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10590-1_28
van der Stap, N., Slump, C.H., Broeders, I.A.M.J., van der Heijden, F.: Image-based navigation for a robotized flexible endoscope. In: Luo, X., Reichl, T., Mirota, D., Soper, T. (eds.) CARE 2014. LNCS, vol. 8899, pp. 77–87. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-13410-9_8
Turan, M., Almalioglu, Y., Araujo, H., Konukoglu, E., Sitti, M.: Deep EndoVO: a recurrent convolutional neural network (RCNN) based visual odometry approach for endoscopic capsule robots. Neurocomputing 275, 1861–1870 (2018)
Turan, M., et al.: Unsupervised odometry and depth learning for endoscopic capsule robots. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1801–1807. IEEE (2018)
Wang, K., Wang, Z., Zhou, Y., Yan, G.: Squirm robot with full bellow skin for colonoscopy. In: 2010 IEEE International Conference on Robotics and Biomimetics, pp. 53–57. IEEE (2010)
Xu, Y., Feng, L., Xia, Z., Xiong, J.: Camera pose estimation based on feature extraction and description for robotic gastrointestinal endoscopy. In: Liu, X.-J., Nie, Z., Yu, J., Xie, F., Song, R. (eds.) ICIRA 2021. LNCS (LNAI), vol. 13015, pp. 113–122. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-89134-3_11
Zhao, Y., Lou, Y.: Vision guided navigation based on dark regions and template matching for capsule endoscopies. In: 2013 IEEE International Conference on Information and Automation (ICIA), pp. 533–538. IEEE (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Tan, M., Feng, L., Xia, Z., Xiong, J. (2022). Deep Motion Flow Estimation for Monocular Endoscope. In: Liu, H., et al. Intelligent Robotics and Applications. ICIRA 2022. Lecture Notes in Computer Science(), vol 13457. Springer, Cham. https://doi.org/10.1007/978-3-031-13835-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-13835-5_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-13834-8
Online ISBN: 978-3-031-13835-5
eBook Packages: Computer ScienceComputer Science (R0)