Skip to main content

Advertisement

SpringerLink
Log in

Social distance control for quadruped robots in a gated spike filter neural network framework

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Ensuring safe human-robot interactions is a crucial concern in robotics research and development. However, controlling the social distance between quadruped robots and humans poses a significant challenge due to the large motion range of robots and the lack of specific social distance rules for human safety. This study proposes an innovative framework that integrates proxemic theory and ISO safety strategy to provide a new social distance control mechanism for human-quadruped robot interactions. The proposed framework uses a gated spike filter neural network (GSN) that fuses LiDAR and RGB-D data to estimate human-robot distance and ensures quantified human-robot social distance with continuous velocity limitations, which current approaches do not offer. The experimental results demonstrate that the GSN algorithm offers high accuracy and is well-suited for large-range human-robot distance control of quadruped robots. It has an average prediction error that is 31.4% lower than traditional Kalman filter methods and offers a more efficient and precise solution for real-time social distance control. Additionally, this study includes a detailed use case that demonstrates how to apply the proposed framework to human-quadruped robot interactions. Overall, this study’s findings have implications for improving the safety and efficiency of human-robot interactions in various applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Availability of data and material

All data generated or analyzed during this study are included in this published article.

Code Availability

The code for the neural network architecture of GSN and a portion of the data are available online at https://gitee.com/shuaizhang_zju/gated-spike-filter-neural-network-framework.git. The entire code for the GSN framework will be released later.

References

  1. Maslow AH (1943) A theory of human motivation. Psychol Rev 50(4):370

    Article  Google Scholar 

  2. Coronado E, Kiyokawa T, Garcia Ricardez GA, Ramirez-Alpizar IG, Venture G, Yamanobe N (2022) Evaluating quality in human-robot interaction: A systematic search and classification of performance and human-centered factors, measures and metrics towards an industry 5.0. J Manuf Syst 63:392–410

    Article  Google Scholar 

  3. Scassellati B, Vázquez M (2020) The potential of socially assistive robots during infectious disease outbreaks. Science Robotics 5(44):eabc9014

  4. Saunderson S, Nejat G (2019) How robots influence humans: A survey of nonverbal communication in social human-robot interaction. International Journal of Social Robotics 11:575–608

    Article  Google Scholar 

  5. Chemweno P, Pintelon L, Decre W (2020) Orienting safety assurance with outcomes of hazard analysis and risk assessment: A review of the iso 15066 standard for collaborative robot systems. Saf Sci 129:104832

    Article  Google Scholar 

  6. Salvini P, Paez-Granados D, Billard A (2021) On the safety of mobile robots serving in public spaces: Identifying gaps in en iso 13482: 2014 and calling for a new standard. ACM Transactions on Human-Robot Interaction (THRI) 10(3):1–27

    Article  Google Scholar 

  7. Scalera L, Giusti A, Vidoni R, Di Cosmo V, Matt D, Riedl M (2020) Application of dynamically scaled safety zones based on the iso/ts 15066: 2016 for collaborative robotics. International Journal of Mechanics and Control 21(1):41–49

    Google Scholar 

  8. Byner C, Matthias B, Ding H (2019) Dynamic speed and separation monitoring for collaborative robot applications-concepts and performance. Robot Comput Integr Manuf 58:239–252

    Article  Google Scholar 

  9. Zhang S, Li S, Li X, Xiong Y, Xie Z (2022) A human-robot dynamic fusion safety algorithm for collaborative operations of cobots. J Intell Robot Syst 104:1–14

    Article  Google Scholar 

  10. Franklin CS, Dominguez EG, Fryman JD, Lewandowski ML (2020) Collaborative robotics: New era of human-robot cooperation in the workplace. J Saf Res 74:153–160

    Article  Google Scholar 

  11. Tanaka Y, Shirai Y, Lin X, Schperberg A, Kato H, Swerdlow A, Kumagai N, Hong D (2022) Scaler: A tough versatile quadruped free-climber robot. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 5632–5639. IEEE

  12. Iqbal A, Gao Y, Gu Y (2020) Provably stabilizing controllers for quadrupedal robot locomotion on dynamic rigid platforms. IEEE/ASME Transactions on Mechatronics 25(4):2035–2044

    Article  Google Scholar 

  13. Hooks J, Ahn MS, Yu J, Zhang X, Zhu T, Chae H, Hong D (2020) Alphred: A multi-modal operations quadruped robot for package delivery applications. IEEE Robotics and Automation Letters 5(4):5409–5416

    Article  Google Scholar 

  14. Cao Z, Hidalgo G, Simon T, Wei S, Sheikh Y (2019) Openpose: Realtime multi-person 2d pose estimation using part affinity fields. IEEE Transactions on Pattern Analysis and Machine Intelligence, pp 1–6

  15. Pathi SK, Kiselev A, Kristoffersson A, Repsilber D, Loutfi A (2019) A novel method for estimating distances from a robot to humans using egocentric rgb camera. Sensors (Basel, Switzerland) 19(14):1–13

    Article  Google Scholar 

  16. Edwards A, Edwards C, Westerman D, Spence PR (2019) Initial expectations, interactions, and beyond with social robots. Comput Hum Behav 90:308–314

    Article  Google Scholar 

  17. Patompak P, Jeong S, Nilkhamhang I, Chong NY (2020) Learning proxemics for personalized human-robot social interaction. International Journal of Social Robotics 12:267–280

    Article  Google Scholar 

  18. Millán-Arias C, Fernandes B, Cruz F (2022) Proxemic behavior in navigation tasks using reinforcement learning. Neural Comput & Applic, pp 1–16

  19. Leichtmann B, Nitsch V (2020) How much distance do humans keep toward robots? literature review, meta-analysis, and theoretical considerations on personal space in human-robot interaction. J Environ Psychol 68:101386

    Article  Google Scholar 

  20. Hall ET (1969) The Hidden Dimension, vol 4 of 10. Anchor Books, New York, 3 edn, 7. An optional note

  21. Gonzalez-Trejo JA, Mercado-Ravell DA, Jaramillo-Avila U (2022) Monitoring social-distance in wide areas during pandemics: a density map and segmentation approach. Appl Intell 52(12):13824–13838

    Article  Google Scholar 

  22. Neggers MME, Cuijpers RH, Ruijten PAM, IJsselsteijn WA (2022) Determining shape and size of personal space of a human when passed by a robot. International Journal of Social Robotics, pp 1–12

  23. Camara F, Fox C (2021) Space invaders: Pedestrian proxemic utility functions and trust zones for autonomous vehicle interactions. International Journal of Social Robotics 13(8):1929–1949

    Article  Google Scholar 

  24. Sprunk C, Lau B, Pfaff P, Burgard W (2017) An accurate and efficient navigation system for omnidirectional robots in industrial environments. Auton Robot 41(2):473–493

    Article  Google Scholar 

  25. Karagiannis P, Kousi N, Michalos G, Dimoulas K, Mparis K, Dimosthenopoulos D, Tokçalar Ö, Guasch T, Gerio GP, Makris S (2022) Adaptive speed and separation monitoring based on switching of safety zones for effective human robot collaboration. Robot Comput Integr Manuf 77:102361

    Article  Google Scholar 

  26. Zhang S, Li S, Wang H, Li X (2022) An intelligent manufacturing cell based on human-robot collaboration of frequent task learning for flexible manufacturing. The International Journal of Advanced Manufacturing Technology 120(9–10):5725-5740

    Article  Google Scholar 

  27. Ferraguti F, Landi CT, Costi S, Bonfè M, Farsoni S, Secchi C, Fantuzzi C (2020) Safety barrier functions and multi-camera tracking for human-robot shared environment. Robot Auton Syst 124:103388

    Article  Google Scholar 

  28. Kolar P, Benavidez P, Mo J (2020) Survey of datafusion techniques for laser and vision based sensor integration for autonomous navigation. Sensors 20(8):2180

    Article  Google Scholar 

  29. Graf F, Odabaşi Ç, Jacobs T, Graf B, Födisch T (2019) Mobika-low-cost mobile robot for human-robot interaction. In 2019 28th IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp 1–6. IEEE

  30. Lee W, Won C, Lim J (2020) Unified calibration for multi-camera multi-lidar systems using a single checkerboard. In 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp 9033–9039

  31. Kummerle J, Kuhner T (2020) Unified intrinsic and extrinsic camera and lidar calibration under uncertainties. In 2020 IEEE International Conference on Robotics and Automation (ICRA), pp 6028–6034

  32. Park K, Kim S, Sohn K (2020) High-precision depth estimation using uncalibrated lidar and stereo fusion. IEEE Trans Intell Transp Syst 21(1):321–335

    Article  Google Scholar 

  33. Li S, Zhang S, Fu Y, Wang H, Han K (2020) Task-based obstacle avoidance for uncertain targets based on semantic object matrix. Control Eng Pract 105:104649

    Article  Google Scholar 

  34. Cheng X, Zhong Y, Dai Y, Ji P, Li H (2019) Noise-aware unsupervised deep lidar-stereo fusion. In 2019 IEEE/CVF Conf Comput Vis Pattern Recognit (CVPR), pp 6332–6341

  35. Narkhede P, Walambe R, Poddar S, Kotecha K (2021) Incremental learning of lstm framework for sensor fusion in attitude estimation. PeerJ Computer Science 7:e662

    Article  Google Scholar 

  36. Shao Z, Cheng G, Ma J, Wang Z, Wang J, Li D (2021) Real-time and accurate uav pedestrian detection for social distancing monitoring in covid-19 pandemic. IEEE transactions on multimedia 24:2069–2083

    Article  Google Scholar 

  37. Pan J, Chitta S, Manocha D (2012) Fcl: A general purpose library for collision and proximity queries. In 2012 IEEE International Conference on Robotics and Automation, pp 3859–3866. IEEE

  38. Munaro M, Basso F, Menegatti E (2012) Tracking people within groups with rgb-d data. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp 2101–2107

  39. Dalal N, Triggs B (2005) Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol 1, pp 886–893

  40. Munaro M, Menegatti E (2014) Fast rgb-d people tracking for service robots. Auton Robot 37(3):227–242

    Article  Google Scholar 

  41. Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In IEEE International Conference on Robotics and Automation (ICRA), Shanghai, China, May 9–13. IEEE

  42. Morgan Q, Ken C, Brian G, Josh F, Tully F, Jeremy L, Rob W, Andrew N (2009) Ros: an open-source robot operating system. In ICRA workshop on open source software, vol 3, p 5. Kobe, Japan

  43. Garrido-Jurado S, Muñoz-Salinas R, Madrid-Cuevas FJ, Marín-Jiménez MJ (2014) Automatic generation and detection of highly reliable fiducial markers under occlusion. Pattern Recogn 47(6):2280–2292

    Article  Google Scholar 

  44. Stampe DM (1993) Heuristic filtering and reliable calibration methods for video-based pupil-tracking systems. Behavior Research Methods, Instruments, & Computers 25(2):137–142

    Article  Google Scholar 

  45. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, Killeen T, Lin Z, Gimelshein N, Antiga L et al (2019) Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems 32:8026–8037

    Google Scholar 

  46. Fujii K (2013) Extended kalman filter. Refernce Manual, pp 14–22

  47. Hornyak V, Vanswearingen JM, Brach JS (2012) Measurement of gait speed. Topics in geriatric rehabilitation 28(1):27–32

    Article  Google Scholar 

Download references

Acknowledgements

This research work received support from the National Natural Science Foundation of China (NO.32200890) and China Postdoctoral Science Foundation (NO.2022M712791).

Author information

Authors and Affiliations

  1. Center for Psychological Sciences, Zhejiang University, Hangzhou, 310063, China

    Shuai Zhang, Yongkai Li, Zehao Huang & Zhiguo Wang

  2. Department of Psychology, Zhejiang Sci-Tech University, Hangzhou, 310000, China

    Rong Wang

Authors
  1. Shuai Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yongkai Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zehao Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Rong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhiguo Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Shuai Zhang and Zhiguo Wang conceived and designed the study. Yongkai Li designed the neural network. Shuai Zhang and Zehao Huang performed the experiments. Shuai Zhang wrote the paper. Shuai Zhang, Zhiguo Wang and Rong Wang reviewed and edited the manuscript. All authors read and approved the manuscript.

Corresponding authors

Correspondence to Shuai Zhang or Zhiguo Wang.

Ethics declarations

Ethics approval

The study was conducted in accordance with the Declaration of Helsinki, with the research protocol approved by the Institutional Review Board at the Center for Psychological Sciences, Zhejiang University (protocol code 2022-008, approved on 13 December 2022).

Consent to participate

Informed consent was obtained from all human subjects to participate in the study.

Consent for publication

Informed consent was obtained from the family to publish the data in an anonymized manner.

Conflict of interest statement

We declare that we have no financial or personal relationships with other people or organizations that influenced our work. There is no professional or other personal interest of any nature or kind in any product, service and/or company that could be construed as influencing the position presented in, or the review of, the manuscript.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Li, Y., Huang, Z. et al. Social distance control for quadruped robots in a gated spike filter neural network framework. Appl Intell 53, 24089–24105 (2023). https://doi.org/10.1007/s10489-023-04832-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04832-w

Keywords

Search

Navigation