Skip to main content
SpringerLink
Log in

A Deep-Learning-based Strategy for Kidnapped Robot Problem in Similar Indoor Environment

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

We present a deep-learning-based strategy that only uses a 2D LiDAR sensor to solve the kidnapped robot problem in similar indoor environments. First, we converted a set of 2D laser data into an RGB-image and an occupancy grid map and stacked them into a multi-channel image. Then, a neural network structure with five convolutional layers and four fully connected layers was designed to regress the 3-DOF robot pose. Finally, the network was trained using multi-channel images as input. We also improved the network structure to identify the scene where the robot is localized. Extensive experiments have been conducted in practice with a real mobile robot, verifying the effectiveness of the proposed strategy. Our network can obtain approximately 2m and 5∘ accuracy indoors, and the scene classification accuracy of our network reaches up to 98%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lingemann, K., Nüchter, A., Hertzberg, J., Surmann, H.: High-speed laser localization for mobile robots. Robotics and Autonomous Systems 51(4), 275–296 (2005)

    Article  Google Scholar 

  2. Csorba, M.: Simultaneous Localisation and Map Building. PhD thesis, University of Oxford Oxford (1997)

  3. Pérez, J., Caballero, F., Merino, L.: Enhanced monte carlo localization with visual place recognition for robust robot localization. Journal of Intelligent & Robotic Systems 80(3-4), 641–656 (2015)

    Article  Google Scholar 

  4. Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics. MIT Press, Cambridge (2005)

    MATH  Google Scholar 

  5. Wang, J., Takahashi, Y.: Slam method based on independent particle filters for landmark mapping and localization for mobile robot based on hf-band rfid system. Journal of Intelligent & Robotic Systems 92 (3-4), 413–433 (2018)

    Article  Google Scholar 

  6. Fox, D., Burgard, W., Dellaert, F., Thrun, S.: Monte Carlo localization: efficient position estimation for mobile robots. AAAI/IAAI 1999(343-349), 2–2 (1999)

    MATH  Google Scholar 

  7. Jensfelt, P., Kristensen, S.: Active global localization for a mobile robot using multiple hypothesis tracking. IEEE Trans. Robot. Autom. 17(5), 748–760 (2001)

    Article  Google Scholar 

  8. Aghili, F., Su, C.-Y.: Robust relative navigation by integration of icp and adaptive kalman filter using laser scanner and imu. IEEE/ASME Transactions on Mechatronics 21(4), 2015–2026 (2016)

    Article  Google Scholar 

  9. Majdik, A., Popa, M., Tamas, L., Szoke, I., Lazea, G.: New approach in solving the kidnapped robot problem. International Symposium on Robotics, pp 1–6 (2010)

  10. Bukhori, I., Ismail, Z.H., Namerikawa, T.: Detection strategy for kidnapped robot problem in landmark-based map monte carlo localization. In: 2015 IEEE International Symposium on Robotics and Intelligent Sensors (IRIS), pp 75–80. IEEE (2015)

  11. Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp 2938–2946 (2015)

  12. Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1–9 (2015)

  13. Kuse, M., Shen, S.: Learning whole-image descriptors for real-time loop detection andkidnap recovery under large viewpoint difference. arXiv:1904.06962 (2019)

  14. Lim, J.H., Leonard, J.J.: Mobile robot relocation from echolocation constraints. IEEE Trans. Pattern Anal. Mach. Intell. 22(9), 1035–1041 (2000)

    Article  Google Scholar 

  15. Yang, G., Chen, Z., Li, Y., Su, Z.: Rapid relocation method for mobile robot based on improved orb-slam2 algorithm. Remote Sens. 11(2), 149 (2019)

    Article  Google Scholar 

  16. Castellanos, J.A., Tardós, J.D., Schmidt, G.: Building a global map of the environment of a mobile robot: the importance of correlations. In: Proceedings of International Conference on Robotics and Automation, vol. 2, pp 1053–1059. IEEE (1997)

  17. Mur-Artal, R., Montiel, J.M.M., Tardos, J.D: Orb-slam: a versatile and accurate monocular slam system. IEEE Transactions on Robotics 31(5), 1147–1163 (2015)

    Article  Google Scholar 

  18. Mur-Artal, R., Tardós, J.D.: Orb-slam2: an open-source slam system for monocular, stereo, and rgb-d cameras. IEEE Trans. Robot. 33(5), 1255–1262 (2017)

    Article  Google Scholar 

  19. Engel, J., Schöps, T., Cremers, D.: Lsd-slam: large-scale direct monocular slam. In: European Conference on Computer Vision, pp 834–849. Springer, Berlin (2014)

  20. Yang, S., Song, Y., Kaess, M., Scherer, S.: Pop-up slam: semantic monocular plane slam for low-texture environments. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 1222–1229. IEEE (2016)

  21. Mur-Artal, R., Tardós, J.D.: Visual-inertial monocular slam with map reuse. IEEE Robotics and Automation Letters 2(2), 796–803 (2017)

    Article  Google Scholar 

  22. Kalman, R.E.: A new approach to linear filtering and prediction problems. Journal of Basic Engineering 82(1), 35–45 (1960)

    Article  MathSciNet  Google Scholar 

  23. Bailey, T., Nieto, J., Guivant, J., Stevens, M., Nebot, E.: Consistency of the ekf-slam algorithm. In: 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 3562–3568. IEEE (2006)

  24. Martinez-Cantin, R., Castellanos, J.A.: Unscented slam for large-scale outdoor environments. In: 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 3427–3432. IEEE (2005)

  25. Sim, R., Elinas, P., Griffin, M., Shyr, A., Little, J.J.: Design and analysis of a framework for real-time vision-based slam using rao-blackwellised particle filters. In: The 3rd Canadian Conference on Computer and Robot Vision (CRV’06), pp 21–21. IEEE (2006)

  26. Robert, C., Casella, G.: Monte Carlo Statistical Methods. Springer, Berlin (2013)

    MATH  Google Scholar 

  27. Zhou, T., Brown, M., Snavely, N., Lowe, D. G: Unsupervised learning of depth and ego-motion from video. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 1851–1858 (2017)

  28. Yin, P., He, Y., Xu, L., Peng, Y., Han, J., Xu, W.: Synchronous adversarial feature learning for lidar based loop closure detection. In: 2018 Annual American Control Conference (ACC), pp 234–239. IEEE (2018)

  29. Chen, Z., Jacobson, A., Sünderhauf, N., Upcroft, B., Liu, L., Shen, C., Reid, I., Milford, M.: Deep learning features at scale for visual place recognition. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp 3223–3230. IEEE (2017)

  30. Li, J., Zhan, H., Chen, B.M., Reid, I., Lee, G.H.: Deep learning for 2d scan matching and loop closure. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 763–768. IEEE (2017)

  31. Kohlbrecher, S., Stryk, O.V., Meyer, J., Klingauf, U.: A flexible: scalable slam system with full 3d motion estimation. In: 2011 IEEE International Symposium on Safety, Security, and Rescue Robotics, pp 155–160. IEEE (2011)

  32. Radwan, A.V.N., Burgard, W.: Vlocnet++: deep multitask learning for semantic visual localization and odometry. IEEE Robotics And Automation Letters (RA-L) 3(4), 4407–4414 (2018)

    Article  Google Scholar 

  33. Valada, N.R.A., Burgard, W.: Deep auxiliary learning for visual localization and odometry. In: Proceedings of the IEEE International Conference on Robotics and Automation (ICRA) (2018)

  34. Cao, J., Bi, Z., Liu, J., Zhao, Z., Su, Y.: A novel relocation method for simultaneous localization and mapping based on deep learning algorithm. Computers & Electrical Engineering 63, 79–90 (2017)

    Article  Google Scholar 

  35. Xu, B., Wang, N., Chen, T., Li, M.: Empirical evaluation of rectified activations in convolutional network. arXiv:1505.00853 (2015)

  36. Fox, D., Thrun, S., Burgard, W., Dellaert, F.: Particle filters for mobile robot localization. In: Sequential Monte Carlo Methods in Practice, pp 401–428. Springer (2001)

  37. Lipowski, A., Lipowska, D.: Roulette-wheel selection via stochastic acceptance. Physica A: Statistical Mechanics and its Applications 391(6), 2193–2196 (2012)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of Intelligent Control and Optimization for Industrial Equipment of Ministry of Education and the School of Control Science and Engineering, Dalian University of Technology, Dalian, 116024, China

    Shikuan Yu, Fei Yan & Yan Zhuang

  2. School of Computer Science and Electronic Engineering, University of Essex, Colchester, UK

    Dongbing Gu

Authors
  1. Shikuan Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dongbing Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fei Yan.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China (U1913201, 61503056) and the Science and Technology Foundation of Liaoning Province of China (20180520031).

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, S., Yan, F., Zhuang, Y. et al. A Deep-Learning-based Strategy for Kidnapped Robot Problem in Similar Indoor Environment. J Intell Robot Syst 100, 765–775 (2020). https://doi.org/10.1007/s10846-020-01216-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-020-01216-x

Keywords

Search

Navigation