Abstract
Gestures and speech modalities play potent roles in social learning, especially in educational settings. Enabling artificial learning companions (i.e., humanoid robots) to perform human-like gestures and speech will facilitate interactive social learning in classrooms. In this paper, we present the implementation of human-generated gestures and speech on the Pepper robot to build a robotic teacher. To this end, we transferred a human teacher gesture to a humanoid robot using a web and a kinect cameras and applied a video-based markerless motion capture technology and an observation-based motion mirroring method. To evaluate the retargeting methods, we presented different types of a humanoid robotic teacher to six teachers and collect their impressions on the practical usage of a robotic teacher in the classroom. Our results show that the presented AI-based open-source gesture retargeting technology was found attractive, as it gives the teachers an agency to design and employ the Pepper robot in their classes. Future work entails the evaluation of our solution to the stakeholders (i.e. teachers) for its usability.
This project was primarily funded by the Deutsche Forschungsgemeinschaft (DFG, German Research Foundation) under Germany’s Excellence Strategy – EXC 2002/1 “Science of Intelligence” – project number 390523135.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bainbridge, W.A., Hart, J.W., Kim, E.S., Scassellati, B.: The benefits of interactions with physically present robots over video-displayed agents. Int. J. Soc. Robot. 3(1), 41–52 (2011)
Belpaeme, T., Kennedy, J., Ramachandran, A., Scassellati, B., Tanaka, F.: Social robots for education: a review. Sci. Robot. 3(21) (2018)
Belpaeme, T., Tanaka, F.: Social robots as educators. In: OECD Digital Education Outlook 2021 Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots: Pushing the Frontiers with Artificial Intelligence, Blockchain and Robots, p. 143 (2021)
Biswas, G., Segedy, J.R., Bunchongchit, K.: From design to implementation to practice a learning by teaching system: Betty’s brain. Int. J. Artif. Intell. Educ. 26(1), 350–364 (2016)
elggem: Github-naoqi-pose-retargeting (2022). https://github.com/elggem/naoqi-pose-retargeting
FraPorta: Pepper openpose teleoperation (2021). https://github.com/FraPorta/pepper_openpose_teleoperation
Kelly, S.D., McDevitt, T., Esch, M.: Brief training with co-speech gesture lends a hand to word learning in a foreign language. Lang. Cogn. Process. 24(2), 313–334 (2009)
Li, J.: The benefit of being physically present: a survey of experimental works comparing copresent robots, telepresent robots and virtual agents. Int. J. Hum Comput Stud. 77, 23–37 (2015)
Newton, D.P., Newton, L.D.: Humanoid robots as teachers and a proposed code of practice. In: Frontiers in Education, vol. 4, p. 125. Frontiers (2019)
Riek, L., Howard, D.: A code of ethics for the human-robot interaction profession. In: Proceedings of We Robot (2014)
Salem, M., Rohlfing, K., Kopp, S., Joublin, F.: A friendly gesture: investigating the effect of multimodal robot behavior in human-robot interaction. In: 2011 Ro-Man, pp. 247–252. IEEE (2011)
Serholt, S., et al.: Teachers’ views on the use of empathic robotic tutors in the classroom. In: The 23rd IEEE International Symposium on Robot and Human Interactive Communication, pp. 955–960. IEEE (2014)
Sidner, C.L., Kidd, C.D., Lee, C., Lesh, N.: Where to look: a study of human-robot engagement. In: Proceedings of the 9th International Conference on Intelligent User Interfaces, pp. 78–84 (2004)
Yun, H.S., Hübert, H., Taliaronak, V., Sardogan, A.: Utilizing machine learning based gesture recognition software, mediapipe, in the context of education and health. Poster presented at: The AI Innovations, Berlin (5-6 May 2022)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Yun, H.S. et al. (2022). AI-Based Open-Source Gesture Retargeting to a Humanoid Teaching Robot. In: Rodrigo, M.M., Matsuda, N., Cristea, A.I., Dimitrova, V. (eds) Artificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners’ and Doctoral Consortium. AIED 2022. Lecture Notes in Computer Science, vol 13356. Springer, Cham. https://doi.org/10.1007/978-3-031-11647-6_51
Download citation
DOI: https://doi.org/10.1007/978-3-031-11647-6_51
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-11646-9
Online ISBN: 978-3-031-11647-6
eBook Packages: Computer ScienceComputer Science (R0)