Skip to main content
SpringerLink
Log in

Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

Navigating in crowd environments is challenging for mobile robots because not only the safety but also the comfort of surrounding pedestrians must be considered. In this work, a deep reinforcement learning framework is introduced for safe and socially compliant robot navigation. We propose a value network for robot decision-making that leverages spatial-temporal reasoning to comprehend crowd interactions. Based on the real-time speed of pedestrians, the hazardous areas that the robot needs to avoid are designed, and a reward function is formulated to guarantee the safety and comfort of pedestrians. Extensive simulation experiments validate that the developed framework outperforms the state-of-the-art methods in terms of success rate (up to 34% increase) and discomfort frequency (up to 54.72% decrease). In addition, real-world experiments illustrate that our approach can predict pedestrian dynamics and navigate the robot safely and reliably in crowds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Algorithm 1
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Kruse T, Pandey AK, Alami R, Kirsch A (2013) Human-aware robot navigation: a survey. Robot Auton Syst 61(12):1726–1743

    Article  Google Scholar 

  2. Hu Z, Zhao Y, Zhang S, Zhou L, Liu J (2022) Crowd-comfort robot navigation among dynamic environment based on social-stressed deep reinforcement learning. Int J Soc Robot 14(4):913–929

    Article  Google Scholar 

  3. Sathyamoorthy AJ, Patel U, Paul M, Kumar NKS, Savle Y, Manocha D (2022) CoMet: Modeling group cohesion for socially compliant robot navigation in crowded scenes. IEEE Robot Autom Lett 7(2):1008–1015

    Article  Google Scholar 

  4. Wang C, Chen X, Li C, Song R, Li Y, Meng MQ-H (2023) Chase and track: Toward safe and smooth trajectory planning for robotic navigation in dynamic environments. IEEE Trans Ind Electron 70(1):604–613

    Article  Google Scholar 

  5. Helbing D, Molnr P (1995) Social force model for pedestrian dynamics. Phys Rev E 51(5):4282–4286

    Article  Google Scholar 

  6. Van Den Berg J, Lin M, Manocha D (2008) Reciprocal velocity obstacles for real-time multi-agent navigation. In: 2008 IEEE International Conference on Robotics and Automation. IEEE, pp. 1928–1935

  7. Van Den Berg J, Guy SJ, Lin M, Manocha D (2011) Reciprocal n-body collision avoidance. Robotics Research. Springer, Berlin, pp 3–19

    Google Scholar 

  8. Trautman P, Krause A (2010) Unfreezing the robot: Navigation in dense, interacting crowds. In: 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 797–803

  9. Sathyamoorthy AJ, Patel U, Guan T, Manocha D (2020) Frozone: freezing-free, pedestrian-friendly navigation in human crowds. IEEE Robot Autom Lett 5(3):4352–4359

    Article  Google Scholar 

  10. Kretzschmar H, Kuderer M, Burgard W (2014) Learning to predict trajectories of cooperatively navigating agents. In: 2014 IEEE International Conference on Robotics and Automation. IEEE, pp. 4015–4020

  11. Pfeiffer M, Schwesinger U, Sommer H, Galceran E, Siegwart R (2016) Predicting actions to act predictably: Cooperative partial motion planning with maximum entropy models. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 2096–2101

  12. Kretzschmar H, Spies M, Sprunk C, Burgard W (2016) Socially compliant mobile robot navigation via inverse reinforcement learning. Int J Robot Res 35(11):1289–1307

    Article  Google Scholar 

  13. Che Y, Okamura AM, Sadigh D (2020) Efficient and trustworthy social navigation via explicit and implicit robot-human communication. IEEE Trans Robot 36(3):692–707

    Article  Google Scholar 

  14. Tai L, Zhang J, Liu M, Burgard W (2018) Socially compliant navigation through raw depth inputs with generative adversarial imitation learning. In: 2018 IEEE International Conference on Robotics and Automation. IEEE, pp. 1111–1117

  15. Long P, Liu W, Pan J (2017) Deep-learned collision avoidance policy for distributed multiagent navigation. IEEE Robot Autom Lett 2(2):656–663

  16. Liu Y, Xu A, Chen Z (2018) Map-based deep imitation learning for obstacle avoidance. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 8644–8649

  17. Chen YF, Everett M, Liu M, How JP (2017) Socially aware motion planning with deep reinforcement learning. In: 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 1343–1350

  18. Cui Y, Zhang H, Wang Y, Xiong R (2021) Learning world transition model for socially aware robot navigation. In: 2021 IEEE International Conference on Robotics and Automation. IEEE, pp. 9262–9268

  19. Chen YF, Liu M, Everett M, How JP (2017) Decentralized non-communicating multiagent collision avoidance with deep reinforcement learning. In: 2017 IEEE International Conference on Robotics and Automation. IEEE, pp. 285–292

  20. Everett M, Chen YF, How JP (2018) Motion planning among dynamic, decision-making agents with deep reinforcement learning. In: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, pp. 3052–3059

  21. Chen C, Liu Y, Kreiss S, Alahi A (2019) Crowd-robot interaction: Crowd-aware robot navigation with attention-based deep reinforcement learning. In: 2019 IEEE International Conference on Robotics and Automation. IEEE, pp. 6015–6022

  22. Bohannon RW (1997) Comfortable and maximum walking speed of adults aged 20–79 years: reference values and determinants. Age Age 26(1):15–19

    Article  Google Scholar 

  23. Samsani SS, Muhammad MS (2021) Socially compliant robot navigation in crowded environment by human behavior resemblance using deep reinforcement learning. IEEE Robot Autom Lett 6(3):5223–5230

    Article  Google Scholar 

  24. Hall ET (1966) The hidden dimension. Anchor, Garden City, NY

    Google Scholar 

  25. Sutton RS, Barto AG (1998) Reinforcement learning: An introduction. MIT Press, Cambridge, MA

    Google Scholar 

Download references

Funding

This work was supported in part by the National Key R & D Program of China under Grant 2017YFB1302400, the Jinan “20 New Colleges and Universities” Funded Scientific Research Leader Studio under Grant 2021GXRC079, the Major Agricultural Applied Technological Innovation Projects of Shandong Province under Grant SD2019NJ014, the Shandong Natural Science Foundation under Grant ZR2019MF064, and the Beijing Advanced Innovation Center for Intelligent Robots and Systems under Grant 2019IRS19.

Author information

Authors and Affiliations

  1. School of Control Science and Engineering, Shandong University, Jinan, 250061, Shandong, China

    Bingxin Xue, Chaoqun Wang & Fengyu Zhou

  2. Academy of Intelligent Innovation, Shandong University, Jinan, 250101, Shandong, China

    Ming Gao

  3. Shandong New Generation Information Industrial Technology Research Institute, Jinan, 250100, Shandong, China

    Yao Cheng

Authors
  1. Bingxin Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chaoqun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yao Cheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Fengyu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fengyu Zhou.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xue, B., Gao, M., Wang, C. et al. Crowd-Aware Socially Compliant Robot Navigation via Deep Reinforcement Learning. Int J of Soc Robotics 16, 197–209 (2024). https://doi.org/10.1007/s12369-023-01071-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-023-01071-4

Keywords

Search

Navigation