Abstract
Indoor global localization is a critical aspect of autonomous robotic navigation. The increasing demand for service consumer-grade robots that require self-localization calls for research on methods that work with easy setup and low-cost sensors. In this paper, we propose a monocular camera-based localization of a motorized wheeled robot using a 2D floor plan as a reference map. The innovation of our method lies in using depth maps estimated from monocular images to compute the free space around the robot to be used as a measurement model in a particle filter strategy. The estimated free space density is compared to the free space density extracted from particles in the 2D floor plan. Due to the inherent imperfections of estimated depth maps, we also propose a new particle weighting approach to account for uncertainties in the depth estimation from the monocular camera. Experiments performed using real-world scenario sequences of images comparing the proposed method with RGB-D camera-based approaches demonstrate the effectiveness of the method, even for imperfect depth maps obtained with the monocular depth estimation model.
Article PDF
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Avoid common mistakes on your manuscript.
Availability of data and materials
The experiments data will be released in a repository in UFRGS Phi Robotics Research Lab GitHub (https://github.com/phir2-lab).
Code Availibility
The code used to run the localization experiments can be found in this repository (https://github.com/phir2-lab/fsd_localization).
References
Ito, S., Endres, F., Kuderer, M., Tipaldi, G.D., Stachniss, C., Burgard, W.: W-rgb-d: Floor-plan-based indoor global localization using a depth camera and wifi, pp. 417–422 (2014)
Boniardi, F., Valada, A., Mohan, R., Caselitz, T., Burgard, W.: Robot localization in floor plans using a room layout edge extraction network, pp. 5291–5297 (2019). https://doi.org/10.1109/IROS40897.2019.8967847
Khan, F., Salahuddin, S., Javidnia, H.: Deep learning-based monocular depth estimation methods–a state-of-the-art review. Sensors 20(8) (2020). https://doi.org/10.3390/s20082272
Maffei, R., Pittol, D., Mantelli, M., Prestes, E., Kolberg, M.: Global localization over 2d floor plans with free-space density based on depth information. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4609–4614 (2020). https://doi.org/10.1109/IROS45743.2020.9340642
Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. Intelligent Robotics and Autonomous Agents series. MIT Press, ??? (2005)
Dellaert, F., Fox, D., Burgard, W., Thrun, S.: Monte carlo localization for mobile robots. In: Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C), vol. 2, pp. 1322–13282 (1999). https://doi.org/10.1109/ROBOT.1999.772544
Alhmiedat, T., Marei, A.M., Messoudi, W., Albelwi, S., Bushnag, A., Bassfar, Z., Alnajjar, F., Elfaki, A.O.: A slam-based localization and navigation system for social robots: The pepper robot case. Machines 11(2) (2023). https://doi.org/10.3390/machines11020158
Boniardi, F., Caselitz, T., Kümmerle, R., Burgard, W.: A pose graphbased localization system for long-term navigation in cad floor plans, vol. 112, pp. 84–97 (2019). https://doi.org/10.1016/j.robot.2018.11.003. https://www.sciencedirect.com/science/article/pii/S0921889018306092
Alhmiedat, T.: Fingerprint-based localization approach for wsn using machine learning models. Appl. Sci. 13(5) (2023). https://doi.org/10.3390/app13053037
Alqobali, R., Alshmrani, M., Alnasser, R., Rashidi, A., Alhmiedat, T., Alia, O.M.: A survey on robot semantic navigation systems for indoor environments. Appl. Sci. 14(1) (2024). https://doi.org/10.3390/app14010089
Winterhalter, W., Fleckenstein, F., Steder, B., Spinello, L., Burgard, W.: Accurate indoor localization for rgb-d smartphones and tablets given 2d floor plans. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3138–3143 (2015). https://doi.org/10.1109/IROS.2015.7353811
Watanabe, Y., Amaro, K.R., Ilhan, B., Kinoshita, T., Bock, T., Cheng, G.: Robust localization with architectural floor plans and depth camera. In: 2020 IEEE/SICE International Symposium on System Integration (SII), pp. 133–138 (2020). https://doi.org/10.1109/SII46433.2020.9025984
Boniardi, F., Caselitz, T., Kümmerle, R., Burgard, W.: Robust lidar-based localization in architectural floor plans. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 3318–3324 (2017). https://doi.org/10.1109/IROS.2017.8206168
Wang, X., Marcotte, R.J., Olson, E.: Glfp: Global localization from a floor plan. In: 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1627–1632 (2019). https://doi.org/10.1109/IROS40897.2019.8968061
Gao, L., Kneip, L.: Fp-loc: Lightweight and drift-free floor plan-assisted lidar localization. In: 2022 International Conference on Robotics and Automation (ICRA), pp. 4142–4148 (2022). https://doi.org/10.1109/ICRA46639.2022.9812361
Ribacki, A., Jorge, V.A.M., Mantelli, M., Maffei, R., Prestes, E.: Visionbased global localization using ceiling space density. In: 2018 IEEE International Conference on Robotics and Automation (ICRA), pp. 3502–3507 (2018). https://doi.org/10.1109/ICRA.2018.8460515
Maffei, R., Jorge, V., Rey, V., Kolberg, M., Prestes, E.: Fast monte carlo localization using spatial density information 2015, 6352–6358 (2015). https://doi.org/10.1109/ICRA.2015.7140091
Borenstein, J., Koren, Y.: Histogramic in-motion mapping for mobile robot obstacle avoidance. IEEE Trans. Robotics Autom. 7, 535–539 (1991)
Ming, Y., Meng, X., Fan, C., Yu, H.: Deep learning for monocular depth estimation: A review. Neurocomputing 438, 14–33 (2021). https://doi.org/10.1016/j.neucom.2020.12.089
Zhao, C., Sun, Q., Zhang, C., Tang, Y., Qian, F.: Monocular depth estimation based on deep learning: An overview. CoRR (2020). arXiv:2003.06620
Kim, Y., Jung, H., Min, D., Sohn, K.: Deep monocular depth estimation via integration of global and local predictions. IEEE Transactions on Image Processing 27, 4131–4144 (2018)
Haji-Esmaeili, M.M., Montazer, G.: Large-scale monocular depth estimation in the wild. Engineering Applications of Artificial Intelligence 127, 107189 (2024). https://doi.org/10.1016/j.engappai.2023.107189
Guizilini, V., Vasiljevic, I., Chen, D., Ambrus, R., Gaidon, A.: Towards zero-shot scale-aware monocular depth estimation, pp. 9199–9209 (2023). https://doi.org/10.1109/ICCV51070.2023.00847
Ranftl, R., Lasinger, K., Hafner, D., Koltun, V.: Towards robust monocular depth estimation: Mixing datasets for zero-shot cross-dataset transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence PP, 1–1 (2020). https://doi.org/10.1109/TPAMI.2020.3019967
Bhat, S.F., Birkl, R., Wofk, D., Wonka, P., Müller, M.: Zoedepth: Zeroshot transfer by combining relative and metric depth (2023). https://doi.org/10.48550/ARXIV.2302.12288
Ranftl, R., Bochkovskiy, A., Koltun, V.: Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pp. 12179–12188 (2021)
Ruiz-Sarmiento, J.R., Galindo, C., González-Jiménez, J.: Robot@home, a robotic dataset for semantic mapping of home environments. Int. J. Robot. Res. 36, 027836491769564 (2017). https://doi.org/10.1177/0278364917695640
Silberman, N., Hoiem, D., Kohli, P., Fergus, R.: Indoor segmentation and support inference from rgbd images. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) Computer Vision – ECCV 2012, pp. 746–760. Springer, Berlin, Heidelberg (2012)
Acknowledgements
Thanks to the personnel from UFRGS Phi Robotics Research Lab. This study was financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001.
Funding
Not applicable.
Author information
Authors and Affiliations
Contributions
Cristian Lopes, Renan Maffei, and Mariana Kolberg conceived and designed the approach. Cristian Lopes carried out the experiments and the data analysis. All authors wrote the manuscript and reviewed its final version.
Corresponding author
Ethics declarations
Ethics approval
Not applicable.
Consent to participate
Not applicable.
Consent for publication
Not applicable.
Conflict of interest/Competing interests
Not applicable.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Open Access This article is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License, which permits any non-commercial use, sharing, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if you modified the licensed material. You do not have permission under this licence to share adapted material derived from this article or parts of it. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by-nc-nd/4.0/.
About this article
Cite this article
Lopes, C., Maffei, R. & Kolberg, M. Monocular Depth Estimation Applied to Global Localization Over 2D Floor Plans Using Free Space Density. J Intell Robot Syst 111, 4 (2025). https://doi.org/10.1007/s10846-024-02131-1
Received:
Accepted:
Published:
DOI: https://doi.org/10.1007/s10846-024-02131-1