Skip to main content
SpringerLink
Log in

A Deep Hierarchical Framework for Robot Global Localization

  • Short Paper
  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Robot global localization is a significant and challenging problem. The popular filter-based methods such as Monte Carlo Localization (MCL), which cover the entire state space with particles, have the problem of high computational cost especially in large-scale environments. In this paper, we propose a deep hierarchical framework for robot 2D global localization, which mainly includes coarse localization and fine localization. The coarse localization takes the RGB image as the input, and it provides an initial probability distribution by using CNN and Long Short-Term Memory (LSTM) based deep network. CNN learns suitable feature representations while LSTM is used for structural dimension reduction and feature association. For fine localization, we present a method combining fast branch-and-bound scan matching (FBBS) and MCL, which takes the pointcloud of LiDAR sensor as the input. FBBS quickly narrows the possible range of actual pose through finding the best match between the laser scan and map. Finally, the precise localization is realized by means of MCL. Finally, we used the Microsoft 7Scenes dataset and the Cambridge Landmarks dataset to evaluate our coarse localization method. It achieves an average accuracy of 0.191 m and 7.96∘ on 7Scenes dataset, and it achieves an average accuracy of 1.17 m and 4.10∘ on Cambridge Landmarks dataset. And the global localization experiments on a real robot shows that our success rate reaches up to 98%. Compared with other state-of-the-art methods, our framework can achieve faster convergence and more accurate estimation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Code Availability

Not applicable.

References

  1. Choi, H., Yang, K.W., Kim, E.: Simultaneous global localization and mapping. IEEE/ASME Trans. Mechatron. 19(4), 1160–1170 (2013)

    Article  Google Scholar 

  2. Gawel, A., Del Don, C., Siegwart, R., Nieto, J., Cadena, C.: X-view: Graph-based semantic multi-view localization. IEEE Robot. Autom. Lett. 3(3), 1687–1694 (2018)

    Article  Google Scholar 

  3. Lam, C.P., Chou, C.T., Chiang, K.H., Fu, L.C.: Human-centered robot navigation—towards a harmoniously human–robot coexisting environment. IEEE Trans. Robot. 27(1), 99–112 (2010)

    Article  Google Scholar 

  4. Fujimori, A., Nikiforuk, P.N., Gupta, M.M.: Adaptive navigation of mobile robots with obstacle avoidance. IEEE Trans. Robot. Autom. 13(4), 596–601 (1997)

    Article  Google Scholar 

  5. Artunedo, A., Villagra, J., Godoy, J., Del Castillo, M.D.: Motion planning approach considering localization uncertainty. IEEE Trans. Veh. Technol. 69(6), 5983–5994 (2020)

    Article  Google Scholar 

  6. Tabasso, C., Mimmo, N., Cichella, V., Marconi, L.: Optimal motion planning for localization of avalanche victims by multiple UAVs. IEEE Contr. Syst. Lett. 5(6), 2054–2059 (2021)

    Article  MathSciNet  Google Scholar 

  7. Zhang, J., Liu, R., Yin, K., Wang, Z., Gui, M., Chen, S.: Intelligent collaborative localization among air-ground robots for industrial environment perception. IEEE Trans. Ind. Electron. 66(12), 9673–9681 (2018)

    Article  Google Scholar 

  8. Zlot, R., Bosse, M.: Efficient large-scale three-dimensional mobile mapping for underground mines. J. Field Robot. 31(5), 758–779 (2014)

    Article  Google Scholar 

  9. Kohlbrecher, S., Von Stryk, O., Meyer, J., Klingauf, U.: A flexible and scalable SLAM system with full 3D motion estimation. In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics (pp. 155-160). IEEE (2011)

  10. Thrun, S., Fox, D., Burgard, W., Dellaert, F.: Robust Monte Carlo localization for mobile robots. Artif. Intell. 128(1–2), 99–141 (2001)

    Article  Google Scholar 

  11. Kucner, T.P., Magnusson, M., Lilienthal, A.J.: Where am I? An NDT-based prior for MCL. In: 2015 European Conference on Mobile Robots (ECMR) (pp. 1-6). IEEE (2015)

  12. Zhang, L., Zapata, R., Lepinay, P.: Self-adaptive Monte Carlo localization for mobile robots using range finders. Robotica 30(2), 229–244 (2012)

    Article  Google Scholar 

  13. Jo, H., Kim, E.: New Monte Carlo localization using deep initialization: A three-dimensional liDAR and a camera fusion approach. IEEE Access 8, 74485–74496 (2020)

    Article  Google Scholar 

  14. Xu, S., Chou, W., Dong, H.: A robust indoor localization system integrating visual localization aided by CNN-based image retrieval with Monte Carlo localization. Sensors 19(2), 249 (2019)

    Article  Google Scholar 

  15. Kim, G., Park, B., Kim, A.: 1-day learning, 1-year localization: Long-term lidar localization using scan context image. IEEE Robot. Autom. Lett. 4(2), 1948–1955 (2019)

    Article  Google Scholar 

  16. Yu, S., Yan, F., Zhuang, Y., Gu, D.: A deep-learning-based strategy for kidnapped robot problem in similar indoor environment. J. Intell. Robot. Syst. 100(3), 765–775 (2020)

    Article  Google Scholar 

  17. Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

  18. Kendall, A., Cipolla, R.: Modelling uncertainty in deep learning for camera relocalization. In: 2016 IEEE International Conference on Robotics and Automation (ICRA) (pp. 4762-4769). IEEE (2016)

  19. Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., Cremers, D.: Image-based localization using lstms for structured feature correlation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 627–637 (2017)

  20. Clark, R., Wang, S., Markham, A., Trigoni, N., Wen, H.: Vidloc: a deep spatio-temporal model for 6-dof video-clip relocalization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6856–6864 (2017)

  21. Melekhov, I., Ylioinas, J., Kannala, J., Rahtu, E.: Image-based localization using hourglass networks. In: Proceedings of the IEEE international conference on computer vision workshops, pp. 879–886 (2017)

  22. Wu, J., Ma, L., Hu, X.: Delving deeper into convolutional neural networks for camera relocalization. In: 2017 IEEE International Conference on Robotics and Automation (ICRA) (pp. 5644-5651). IEEE (2017)

  23. Naseer, T., Burgard, W.: Deep regression for monocular camera-based 6-dof global localization in outdoor environments (2017)

  24. Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5974–5983 (2017)

  25. Yee, M. Y., Vermaak, J.: A grid-based proposal for efficient global localisation of mobile robots. In: Proceedings.(ICASSP’05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005. (Vol. 5, pp. v-217) IEEE (2005)

  26. Chowdhary, G., Johnson, E.N., Magree, D., Wu, A., Shein, A.: GPS-Denied indoor and outdoor monocular vision aided navigation and control of unmanned aircraft. J. Field Robot. 30(3), 415–438 (2013)

    Article  Google Scholar 

  27. Perea, D., Hernández-Aceituno, J., Morell, A., Toledo, J., Hamilton, A., Acosta, L.: MCL with sensor fusion based on a weighting mechanism versus a particle generation approach. In: 16th International IEEE Conference on Intelligent Transportation Systems (ITSC 2013) (pp. 166-171). IEEE (2013)

  28. Perea-Strom, D., Morell, A., Toledo, J., Acosta, L.: GNSS Integration in the localization system of an autonomous vehicle based on particle weighting. IEEE Sensors J. 20(6), 3314–3323 (2019)

    Article  Google Scholar 

  29. He, T., Hirose, S.: Observation-driven Bayesian filtering for global location estimation in the field area. J Field Robot 30(4), 489–518 (2013)

    Article  Google Scholar 

  30. Jianchao, S., Xuebo, Z., Lei, S., Jingtai, L.: Map-based robust localization for indoor mobile robots. In: 2017 36th Chinese Control Conference (CCC) (pp. 6945-6950). IEEE (2017)

  31. Schmiedel, T., Einhorn, E., Gross, H. M.: IRON: A fast interest point descriptor for robust NDT-map matching and its application to robot localization. In: 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (pp. 3144-3151). IEEE (2015)

  32. Hess, W., Kohler, D., Rapp, H., Andor, D.: Real-time loop closure in 2D LIDAR SLAM. In: 2016 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1271-1278). IEEE (2016)

  33. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778 (2016)

  34. Shotton, J., Glocker, B., Zach, C., Izadi, S., Criminisi, A., Fitzgibbon, A.: Scene coordinate regression forests for camera relocalization in RGB-d images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2930–2937 (2013)

  35. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the thirteenth international conference on artificial intelligence and statistics, pp. 249–256 (2010)

  36. Kim, G., Kim, A.: Scan context: Egocentric spatial descriptor for place recognition within 3d point cloud map. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4802–4809 (2018)

Download references

Funding

This work was supported by National Key Research and Development Program of China (Grant No. 2021YFF0307900).

Author information

Authors and Affiliations

  1. School of Mechanical Engineering, Shanghai Jiao Tong University, 800 Dongchuan Road, Shanghai, 200240, Shanghai, China

    Zelin Wang, Feng Gao, Yue Zhao, Yunpeng Yin & Liangyu Wang

Authors
  1. Zelin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Feng Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yue Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yunpeng Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liangyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Zelin Wang, Feng Gao and Yue Zhao contributed to the study conception and design. Material preparation, data collection, experiments and analysis were performed by Zelin Wang, Yue Zhao, Yunpeng Yin and Liangyu Wang. The first draft of the manuscript was written by Zelin Wang and all authors commented on previous versions of the manuscript. All authors read and approved the final manuscript.

Corresponding author

Correspondence to Feng Gao.

Ethics declarations

Ethics approval

Not applicable.

Conflict of Interests

Not applicable.

Consent for Publication

Not applicable.

Consent to participate

Not applicable.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, Z., Gao, F., Zhao, Y. et al. A Deep Hierarchical Framework for Robot Global Localization. J Intell Robot Syst 106, 46 (2022). https://doi.org/10.1007/s10846-022-01728-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s10846-022-01728-8

Keywords

Search

Navigation