Abstract
Human-robot interaction (HRI) is crucial for fostering seamless and effective collaboration between humans and robots, particularly in Industry 4.0 and related fields. This study investigates the integration of large language models (LLMs) to enhance HRI by enabling natural language understanding and efficient task execution planning. We propose a novel approach that leverages LLMs to facilitate seamless communication between users and robotic systems. The system interprets user intentions conveyed in plain text, plans robotic actions, and executes tasks in real-world scenarios. Through an experimental case study, we validate the effectiveness of this approach. The results vividly underscore the transformative potential of LLMs in bridging the gap between natural language commands and robotic actions, thereby significantly advancing applications in industrial automation and beyond.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Proia, S., Carli, R., Cavone, G., Dotoli, M.: Control techniques for safe, ergonomic, and efficient human-robot collaboration in the digital industry: a survey. IEEE Trans. Autom. Sci. Eng. 19(3), 1798–1819 (2021)
Villani, V., Pini, F., Leali, F., Secchi, C.: Survey on human-robot collaboration in industrial settings: safety, intuitive interfaces and applications. Mechatronics 55, 248–266 (2018)
Wang, C., et al.: LaMI: large language models for multi-modal human-robot interaction. In: Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, pp. 1–10 (2024)
Su, H., Qi, W., Chen, J., Yang, C., Sandoval, J., Amine Laribi, M.: Recent advancements in multimodal human–robot interaction. Front. Neurorob. 17, 1084000 (2023)
Chryssolouris, G., Alexopoulos, K., Arkouli, Z.: A Perspective on Artificial Intelligence in Manufacturing, vol. 436. Springer, Cham (2023). https://doi.org/10.1007/978-3-031-21828-6
Zhang, Z., Chai, W., Wang, J.: Mani-GPT: a generative model for interactive robotic manipulation. Procedia Comput. Sci. 226, 149–156 (2023)
Hicks, M.T., Humphries, J., Slater, J.: ChatGPT is bullshit. Ethics Inf. Technol. 26(2), 38 (2024)
Asai, A., Min, S., Zhong, Z., Chen, D.: Retrieval-based language models and applications. In: Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 6: Tutorial Abstracts), pp. 41–46 (2023)
Ahn, M., et al.: Do as I can, not as I say: grounding language in robotic affordances. arXiv preprint arXiv:2204.01691 (2022)
Gkournelos, C., Konstantinou, C., Makris, S.: An LLM-based approach for enabling seamless human-robot collaboration in assembly. CIRP Ann. (2024)
Han, Y., Liu, C., Wang, P.: A comprehensive survey on vector database: storage and retrieval technique, challenge. arXiv preprint arXiv:2310.11703 (2023)
Tanneberg, D., et al.: To help or not to help: LLM-based attentive support for human-robot group interactions. arXiv preprint arXiv:2403.12533 (2024)
Calanzone, D., Coppari, A., Tedoldi, R., Olivato, G., Casonato, C.: An open source perspective on AI and alignment with the EU AI act (2023)
Mignone, G., Parziale, A., Ferrentino, E., Marcelli, A., Chiacchio, P.: Observation vs. interaction in the recognition of human-like movements. Front. Rob. AI 10 (2023)
Mavridis, N.: A review of verbal and non-verbal human-robot interactive communication. Robot. Auton. Syst. 63, 22–35 (2015)
Horswill, I.: Polly: a vision-based artificial agent. In: AAAI, pp. 824–829 (1993)
Fry, J., Asoh, H., Matsui, T.: Natural dialogue with the Jijo-2 office robot. In: Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No. 98CH36190), vol. 2, pp. 1278–1283. IEEE (1998)
Crangle, C., Suppes, P.: Language and learning for robots. Number 41. Center for the Study of Language (CSLI) (1994)
Searle, J.R.: Speech Acts: An Essay in the Philosophy of Language, vol. 626. Cambridge University Press (1969)
Bicho, E., Louro, L., Erlhagen, W.: Integrating verbal and nonverbal communication in a dynamic neural field architecture for human-robot interaction. Front. Neurorobot. 4, 1222 (2010)
Zhang, B., Soh, H.: Large language models as zero-shot human models for human-robot interaction. In: 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 7961–7968. IEEE (2023)
Yoshida, T., Masumori, A., Ikegami, T.: From text to motion: grounding GPT-4 in a humanoid robot" alter3". arXiv preprint arXiv:2312.06571 (2023)
Bärmann, L., Kartmann, R., Peller-Konrad, F., Waibel, A., Asfour, T.: Incremental learning of humanoid robot behavior from natural interaction and large language models. arXiv preprint arXiv:2309.04316 (2023)
OpenAI: Hello GPT-4o, May 2024. https://openai.com/index/hello-gpt-4o/
Huang, X., et al.: Understanding the planning of LLM agents: a survey (2024)
Lewis, P., et al.: Retrieval-augmented generation for knowledge-intensive NLP tasks (2021)
Rackauckas, Z.: RAG-Fusion: a new take on retrieval augmented generation. Int. J. Nat. Lang. Comput. 13(1), 37–47 (2024)
Yang, L., et al.: Buffer of thoughts: thought-augmented reasoning with large language models (2024)
SERMAS-EU. Sermas toolkit. https://sermas-eu.github.io/
Macenski, S., Foote, T., Gerkey, B., Lalancette, C., Woodall, W.: Robot operating system 2: design, architecture, and uses in the wild. Sci. Rob. 7(66), eabm6074 (2022)
Acknowledgments
This work was supported by the Socially-acceptable Extended Reality Models and Systems (SERMAS) Project of the European Union’s Horizon Europe Research and Innovation Program (GA n. 101070351).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2025 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Coppari, A. et al. (2025). A Large Language Model-Based Motion Planning for Human-Robot Interaction: An Experimental Case Study. In: Paolillo, A., Giusti, A., Abbate, G. (eds) Human-Friendly Robotics 2024. HFR 2024. Springer Proceedings in Advanced Robotics, vol 35. Springer, Cham. https://doi.org/10.1007/978-3-031-81688-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-031-81688-8_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-81687-1
Online ISBN: 978-3-031-81688-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)