Skip to main content
SpringerLink
Log in

A Multimodal Path Planning Approach to Human Robot Interaction Based on Integrating Action Modeling

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

To complete a task consisting of a series of actions that involve human-robot interaction, it is necessary to plan a motion that considers each action individually as well as in relation to the following action. We then focus on the specific action of “approaching a group of people” in order to accurately obtain human data that is used to make the performance of tasks involving interactions with multiple people more smooth. The movement depends on the characteristics of the important sensors used for the task and on the placement of people at and around the destination. Considering the multiple tasks and placement of people, the pre-calculation of the destinations and paths is difficult. This paper thus presents a system of navigation that can accurately obtain human data based on sensor characteristics, task content, and real-time sensor data for processes involving human-robot interaction (HRI); this method does not navigate specifically toward a previously determined static point. Our goal was achieved by using a multimodal path planning based on integration of action modeling by considering both voice and image sensing of interacting people as well as obstacle avoidance. We experimentally verified our method by using a robot in a coffee shop environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. McColl, D., Hong, A., Hatakeyama, N., Nejat, G., Benhabib, B.: A Survey of Autonomous Human Affect Detection Methods for Social Robots Engaged in Natural HRI. J. Intell. Robot. Syst. (2016)

  2. Dantam, N.T., Chaudhuri, S., Kavraki, L.E.: The Task-Motion Kit: An Open Source, General-Purpose Task and Motion-Planning Framework. IEEE Robot. Autom. Mag. (2018)

  3. Park, Y.S., Cho, H.S.: Task Oriented Optimum Positioning of a Mobile Manipulator Base in a Cluttered Environment. J. Intell. Robot. Syst. (1997)

  4. Cognetti, M., Mohammadi, P., Oriolo, G., Vendittelli, M.: Task-oriented whole-body planning for humanoids based on hybrid motion generation, pp 4071–4076 (2014)

  5. Lagriffoul, F., Dimitrov, D., Bidot, J., Saffiotti, A., Karlsson, L.: Efficiently combining task and motion planning using geometric constraints. Int. J. Robot. Res. (2014)

  6. Long, X., Wonsick, M., Dimitrov, V., Padir, T.: Anytime multi-task motion planning for humanoid robots (2017)

  7. Sisbot, E.A., Sisbot, L.F., Alami, T.: A Human Aware Mobile Robot Motion Planner. IEEE T. Robot. (2007)

  8. Rios-Martinez, J., Spalanzani, A., Laugier, C.: Understanding human interaction for probabilistic autonomous navigation using risk-RRT approach. IEEE Int. C. Int. Robot. (2011)

  9. Batista, M.R.: Socially Acceptable Navigation of People with Multi-robot Teams. J. Intell. Robot. Syst. (2019)

  10. Kruse, T., Pandey, A.K., Alami, R., Kirsch, A.: Human-aware robot navigation: A survey. Robot. Auton. Syst. (2013)

  11. Lu, D.V., Hershberger, D., Smart, W.D.: Layered costmaps for context-sensitive navigation. IEEE Int. C. Int. Robot. (2014)

  12. Kollmitz, M., Hsiao, K., Gaa, J., Burgard, W.: Time dependent planning on a layered social cost map for human-aware robot navigation. European Conference on Mobile Robots (2015)

  13. Charalampous, K., Kostavelis, I., Gasteratos, A.: Recent trends in social aware robot navigation: A survey. Robot. Auton. Syst. (2017)

  14. Rios-Martinez, J., Escobedo, A.: Intention Driven Human Aware Navigation for Assisted Mobility. Workshop on Assistance and Service Robotics in a Human Environment at International Conference on Intelligent Robots and Systems (2012)

  15. Hirakawa, T., Yamashita, T., Fujiyoshi, H.: Scene Context-aware Rapidly-exploring Random Trees for Global Path Planning. IEEE International Conference on Pervasive Computing and Communications Workshops (2019)

  16. Dautenhahn, K., Walters, M., Woods, S., Koay, K.L., Nehaniv, C.L., Sisbot, E.A., Alami, R., Siméon, T.: How may I serve you? A robot companion approaching a seated person in a helping context. ACMIEEE Int. Conf. Hum, pp 172–179 (2006)

  17. Koay, K.L., Sisbot, E.A., Syrdal, D.S., Walters, M.L., Dautenhahn, K., Alami, R.: Exploratory study of a robot approaching a person in the context of handing over an object. AAAI spring symposium: multidisciplinary collaboration for socially assistive robotics, pp 18–24 (2007)

  18. Satake, S., Kanda, T., Glas, D.F., Imai, M., Ishiguro, H., Hagita, N.: How to approach humans?-Strategies for social robots to initiate interaction. ACMIEEE Int. Conf. Hum. (2008)

  19. Kato, Y., Kanda, T., Ishiguro, H.: May i help you?: Design of Human-like Polite Approaching Behavior. ACMIEEE Int. Conf. Hum. (2015)

  20. Mizumaru, K., Satake, S., Kanda, T., Ono, T.: Stop Doing it! Approaching Strategy for a Robot to Admonish Pedestrians. ACMIEEE Int. Conf. Hum. (2019)

  21. Mead, R., Matarić, M.J.: Autonomous human robot proxemics: socially aware navigation based on interaction potential. Auton. Robot. (2017)

  22. Cao, Z., Simon, T., Wei, S., Sheikh, Y.: Realtime Multi-Person 2D Pose Estimation using Part Affinity Fields, pp 7291–7299 (2016)

  23. CMU-Perceptual-Computing-Lab: OpenPose. https://github.com/CMU-Perceptual-Computing-Lab/openpose, Accessed 11 December 2019 (2017)

  24. Khoshelham, K, Elberink, S.O.: Accuracy and resolution of kinect depth data for indoor mapping applications. SENSORS 12(2), 1437–1454 (2012). https://doi.org/10.3390/s120201437

    Article  Google Scholar 

  25. Ishiguro, H., Miyashita, T., Kanda, T.: Science of knowledge Communication robot, Ohmsha (2005)

  26. Lewis, J.: Microphone specifications explained, Application Note 1112, Analog Devices (2011)

  27. Tsuzaki, R., Yoshida, K.: Motion Control Based on Fuzzy Potential Method for Autonomous Mobile Robot with Omnidirectional Vision. Journal of the Robotics Society of Japan 6, 676–682 (2003)

    Google Scholar 

  28. Suzuki, T., Takahashi, M.: Obstacle Avoidance for Autonomous Mobile Robots based on Position Prediction using Fuzzy Inference. In: ICINCO-RA, pp 299–304 (2009)

  29. Takahashi, M., Suzuki, T., Cinquegrani, F., Sorbello, R., Pagello, E.: A mobile robot for transport applications in hospital domain with safe human detection algorithm. IEEE Robio., pp 1543–1548 (2009)

  30. ROS.org: amcl. http://wiki.ros.org/amcl, Accessed 11 December 2019 (2019)

  31. ROS.org: gmapping. http://wiki.ros.org/gmapping, Accessed 11 December 2019 (2019)

Download references

Acknowledgments

This study was supported by “A Framework PRINTEPS to Develop Practical Artificial Intelligence” of the Core Research for Evolutional Science and Technology (CREST) of the Japan Science and Technology Agency (JST) under Grant Number JPMJCR14E3.

Author information

Authors and Affiliations

  1. The Graduate School of Science and Technology, Keio University, 3-14-1 Hiyoshi, Kohoku-ku, Yokohama, 223-8522, Japan

    Yosuke Kawasaki & Ayanori Yorozu

  2. The Department of System Design Engineering, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama, 223-8522, Japan

    Masaki Takahashi

  3. Intelligent Autonomous System Lab, Department of Information Engineering, University of Padova, Padua, Italy, and IT+Robotics Srl, Vicenza, Italy

    Enrico Pagello

Authors
  1. Yosuke Kawasaki
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ayanori Yorozu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Masaki Takahashi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Enrico Pagello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yosuke Kawasaki.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

(MP4 140 MB)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kawasaki, Y., Yorozu, A., Takahashi, M. et al. A Multimodal Path Planning Approach to Human Robot Interaction Based on Integrating Action Modeling. J Intell Robot Syst 100, 955–972 (2020). https://doi.org/10.1007/s10846-020-01244-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-020-01244-7

Keywords

Search

Navigation