Skip to main content
SpringerLink
Log in

Person-Following Algorithm Based on Laser Range Finder and Monocular Camera Data Fusion for a Wheeled Autonomous Mobile Robot

  • Conference paper
  • First Online:
Book cover Interactive Collaborative Robotics (ICR 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12336))

Included in the following conference series:

Abstract

Reliable human following is one of the key capabilities of service and personal assisting robots. This paper presents a novel person tracking and following approach for autonomous mobile robots that are equipped with a 2D laser rangefinder (LRF) and a monocular camera. The proposed method does not impose restrictions on a person’s clothes, does not require a head or an upper body to be within a camera field of view and is suitable for low height indoor robots as well. The algorithm is based on a metric that takes into an account parameters obtained directly from LRF and monocular camera data. The algorithm was implemented and tested in the Gazebo simulator. Next, it was integrated into a control system of the TIAGo Base mobile robot and successfully validated in university environment experiments with real people. In addition, this paper proposes a new criterion of algorithm performance estimation, which is a function of false positives number and traveled distances by a person and by a robot. Further this criterion is used to compare performance of the proposed method with the Multiple Instance Learning (MIL) tracker in simulated and in real world environments.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. PMB-2 technical specifications. http://pal-robotics.com/wp-content/uploads/2016/07/PMB-2-Datasheet.pdf

  2. Tiago base. http://wiki.ros.org/Robots/PMB-2

  3. Abbyasov, B., Lavrenov, R., Zakiev, A., Yakovlev, K., Svinin, M., Magid, E.: Automatic tool for gazebo world construction: from a grayscale image to a 3D solid model. In: International Conference on Robotics and Automation (ICRA), pp. 7226–7232 (2020)

    Google Scholar 

  4. Arras, K.O., et al.: Range-based people detection and tracking for socially enabled service robots. In: Prassler, E., et al. (eds.) Towards Service Robots for Everyday Environments, pp. 235–280. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-25116-0_18

    Chapter  Google Scholar 

  5. Babenko, B., Yang, M., Belongie, S.: Visual tracking with online multiple instance learning. In: Conference on Computer Vision and Pattern Recognition, pp. 983–990. IEEE (2009)

    Google Scholar 

  6. Bereznikov, D., Zakiev, A.: Network failure detection and autonomous return for PMB-2 mobile robot. In: International Conference on Artificial Life and Robotics (ICAROB 2020), pp. 444–447 (2020)

    Google Scholar 

  7. Bohannon, R.: Comfortable and maximum walking speed of adults aged 20–79 years: Reference values and determinants. Age Ageing 26(1), 15–19 (1997)

    Article  Google Scholar 

  8. Carballo, A., Ohya, A., Yuta, S.: Reliable people detection using range and intensity data from multiple layers of laser range finders on a mobile robot. Int. J. Soc. Robot. 3, 167–186 (2011)

    Article  Google Scholar 

  9. Chebotareva, E., Hsia, K.H., Yakovlev, K., Magid, E.: Laser rangefinder and monocular camera data fusion for human-following algorithm by PMB-2 mobile robot in simulated Gazebo environment. Smart Innovation, Syst. Technol. 187 (2020)

    Google Scholar 

  10. Chen, B.X., Sahdev, R., Tsotsos, J.K.: Integrating stereo vision with a CNN tracker for a person-following robot. In: Liu, M., Chen, H., Vincze, M. (eds.) ICVS 2017. LNCS, vol. 10528, pp. 300–313. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-68345-4_27

    Chapter  Google Scholar 

  11. Franěk, M.: Environmental factors influencing pedestrian walking speed. Percept. Mot. Skills 116(3), 992–1019 (2013)

    Article  Google Scholar 

  12. Guerrero-Higueras, Á.M., et al.: Tracking people in a mobile robot from 2D LIDAR scans using full convolutional neural networks for security in cluttered environments. Front. Neurorobot. 12 (2018)

    Google Scholar 

  13. Islam, M.J., Hong, J., Sattar, J.: Person-following by autonomous robots: a categorical overview. The Int. J. Robot. Res. 38(14), 1581–1618 (2019)

    Article  Google Scholar 

  14. Islam, M.M., Lam, A., Fukuda, H., Kobayashi, Y., Kuno, Y.: An intelligent shopping support robot: understanding shopping behavior from 2D skeleton data using GRU network. ROBOMECH J. 6(1), 1–10 (2019). https://doi.org/10.1186/s40648-019-0150-1

    Article  Google Scholar 

  15. Jiang, S., Li, L., Hang, M., Kuc, T.: An adaptive 2D tracking approach for person following robot. In: International Symposium on Computer Science and Intelligent Controls, pp. 147–151 (2017)

    Google Scholar 

  16. Kawarazaki, N., et al.: Development of human following mobile robot system using laser range scanner. Procedia Comput. Sci. 76, 455–460 (2015)

    Article  Google Scholar 

  17. Kim, H., et al.: Sensor fusion-based human tracking using particle filter and data mapping analysis in in/outdoor environment. In: International Conference on Ubiquitous Robots and Ambient Intelligence, pp. 741–744 (2013)

    Google Scholar 

  18. Koide, K., et al.: Monocular person tracking and identification with on-line deep feature selection for person following robots. Robot. Auton. Syst. 124 (2020)

    Google Scholar 

  19. Kristou, M., et al.: Target person identification and following based on omnidirectional camera and LRF data fusion. In: International Conference on Robot & Human Interactive Communication, pp. 419–424. IEEE (2011)

    Google Scholar 

  20. Lang, A., Vora, S., Caesar, H., Zhou, L., Yang, J., Beijbom, O.: Pointpillars: fast encoders for object detection from point clouds. In: Conference on Computer Vision and Pattern Recognition, pp. 12689–12697 (2019)

    Google Scholar 

  21. Lavrenov, R., Matsuno, F., Magid, E.: Modified spline-based navigation: guaranteed safety for obstacle avoidance. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2017. LNCS (LNAI), vol. 10459, pp. 123–133. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66471-2_14

    Chapter  Google Scholar 

  22. Lavrenov, R.O., Magid, E.A., Matsuno, F., Svinin, M.M., Suthakorn, J.: Development and implementation of spline-based path planning algorithm in ROS/gazebo environment. Trudy SPIIRAN 18(1), 57–84 (2019)

    Google Scholar 

  23. Lee, B.J., et al.: Robust human following by deep Bayesian trajectory prediction for home service robots. In: International Conference on Robotics and Automation, pp. 7189–7195. IEEE (2018)

    Google Scholar 

  24. Leigh, A., et al.: Person tracking and following with 2D laser scanners. In: International Conference on Robotics and Automation, pp. 726–733. IEEE (2015)

    Google Scholar 

  25. Magid, E., Lavrenov, R., Khasianov, A.: Modified spline-based path planning for autonomous ground vehicle. ICINCO 2, 132–141 (2017)

    Google Scholar 

  26. Moskvin, I., Lavrenov, R.: Modeling tracks and controller for servosila engineer robot. In: Ronzhin, A., Shishlakov, V. (eds.) Proceedings of 14th International Conference on Electromechanics and Robotics “Zavalishin’s Readings”. SIST, vol. 154, pp. 411–422. Springer, Singapore (2020). https://doi.org/10.1007/978-981-13-9267-2_33

    Chapter  Google Scholar 

  27. Nakamori, Y., Hiroi, Y., Ito, A.: Multiple player detection and tracking method using a laser range finder for a robot that plays with human. ROBOMECH Journal 5(1), 1–15 (2018). https://doi.org/10.1186/s40648-018-0122-x

    Article  Google Scholar 

  28. Orita, Y., Fukao, T.: Robust human tracking of a crawler robot. J. Robot. Mechatron. 31(2), 194–202 (2019)

    Article  Google Scholar 

  29. Pages, J., Marchionni, L., Ferro, F.: Tiago: the modular robot that adapts to different research needs. In: International Workshop on Robot Modularity, IROS (2016)

    Google Scholar 

  30. Ren, Q., et al.: Real-time target tracking system for person-following robot. In: Chinese Control Conference, pp. 6160–6165 (2016)

    Google Scholar 

  31. Ronzhin, A., Saveliev, A., Basov, O., Solyonyj, S.: Conceptual model of cyberphysical environment based on collaborative work of distributed means and mobile robots. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds.) ICR 2016. LNCS (LNAI), vol. 9812, pp. 32–39. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-43955-6_5

    Chapter  Google Scholar 

  32. Safin, R., Lavrenov, R., Tsoy, T., Svinin, M., Magid, E.: Real-time video server implementation for a mobile robot. In: 11th International Conference on Developments in eSystems Engineering (DeSE), pp. 180–185. IEEE (2018)

    Google Scholar 

  33. Sato, Y., et al.: A maneuverable robotic wheelchair able to move adaptively with a caregiver by considering the situation. In: International Conference on Robot & Human Interactive Communication, pp. 282–287. IEEE (2013)

    Google Scholar 

  34. Simakov, N., Lavrenov, R., Zakiev, A., Safin, R., Martínez-García, E.A.: Modeling USAR maps for the collection of information on the state of the environment. In: 2019 12th International Conference on Developments in eSystems Engineering (DeSE), pp. 918–923. IEEE (2019)

    Google Scholar 

  35. Sung, Y., Chung, W.: Hierarchical sample-based joint probabilistic data association filter for following human legs using a mobile robot in a cluttered environment. IEEE Trans. Hum. Mach. Syst. 46(3), 340–349 (2015)

    Article  Google Scholar 

  36. Yan, Y., Mao, Y., Li, B.: SECOND: sparsely embedded convolutional detection. Sensors 18, 3337 (2018)

    Article  Google Scholar 

Download references

Acknowledgements

The reported study was funded by the Russian Foundation for Basic Research (RFBR) according to the research project No. 19–58-70002. Special thanks to PAL Robotics for their kind professional support with TIAGo Base robot software and hardware related issues.

Author information

Authors and Affiliations

  1. Laboratory of Intelligent Robotic Systems (LIRS), Intelligent Robotics Department, Higher Institute for Information Technology and Intelligent Systems, Kazan Federal University, Kazan, Russian Federation

    Elvira Chebotareva, Ramil Safin & Evgeni Magid

  2. Department of Electrical Engineering, National Yunlin University of Science and Technology, Douliu, Taiwan

    Kuo-Hsien Hsia

  3. Institute of Innovation for Future Society, Nagoya University, Nagoya, Japan

    Alexander Carballo

Authors
  1. Elvira Chebotareva
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ramil Safin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kuo-Hsien Hsia
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Alexander Carballo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Evgeni Magid
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Elvira Chebotareva .

Editor information

Editors and Affiliations

  1. St. Petersburg Federal Research Center of the Russian Academy of Sciences, St. Petersburg, Russia

    Andrey Ronzhin

  2. Technical University of Munich, Munich, Germany

    Gerhard Rigoll

  3. Institute of Control Sciences of the Russian Academy of Sciences, Moscow, Russia

    Roman Meshcheryakov

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Chebotareva, E., Safin, R., Hsia, KH., Carballo, A., Magid, E. (2020). Person-Following Algorithm Based on Laser Range Finder and Monocular Camera Data Fusion for a Wheeled Autonomous Mobile Robot. In: Ronzhin, A., Rigoll, G., Meshcheryakov, R. (eds) Interactive Collaborative Robotics. ICR 2020. Lecture Notes in Computer Science(), vol 12336. Springer, Cham. https://doi.org/10.1007/978-3-030-60337-3_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60337-3_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60336-6

  • Online ISBN: 978-3-030-60337-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation