Skip to main content
SpringerLink
Log in

Personalized Storytelling with Social Robot Haru

  • Conference paper
  • First Online:
Social Robotics (ICSR 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13818))

Included in the following conference series:

Abstract

In previous studies of applying storytelling to robotics, the emotions and actions of robots are usually pre-determined, resulting in a homogeneous storytelling style for the robot. In this paper, we propose an empathic and adaptive framework for robot’s storytelling that facilitates social robot Haru to learn from human teachers. In this framework, the robot Haru performs empathic storytelling based on the human teacher’s voice, and then changes its narrative styles (e.g., featured by pitch, emotion, action, etc.) to capture the listener’s attention. The whole experiment was conducted on social robot Haru. Haru’s communicative modality involves face and body movements, sound voice and non-verbal sound, which have great potentials for storytelling. The affective robot for storytelling was compared to a neural one and human teachers. Preliminary results show the social robot for storytelling can make use of human teachers as integral to the design of the system and provide a personalized storytelling experience. Moreover, participants had positive attitudes toward storytelling by an affective robot compared to a neutral one.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.unicef.org/globalinsight/media/2206/file.

References

  1. Appel, M., Lugrin, B., Kühle, M., Heindl, C.: The emotional robotic storyteller: on the influence of affect congruency on narrative transportation, robot perception, and persuasion. Comput. Hum. Behav. 120, 106749 (2021)

    Article  Google Scholar 

  2. Azuar, D., Gallud, G., Escalona, F., Gomez-Donoso, F., Cazorla, M.: A story-telling social robot with emotion recognition capabilities for the intellectually challenged. In: Silva, M.F., Luís Lima, J., Reis, L.P., Sanfeliu, A., Tardioli, D. (eds.) ROBOT 2019. AISC, vol. 1093, pp. 599–609. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-36150-1_49

    Chapter  Google Scholar 

  3. Bänziger, T., Scherer, K.R.: The role of intonation in emotional expressions. Speech Commun. 46(3–4), 252–267 (2005)

    Article  Google Scholar 

  4. Cabibihan, J.J., So, W.C., Pramanik, S.: Human-recognizable robotic gestures. IEEE Trans. Auton. Ment. Dev. 4(4), 305–314 (2012)

    Article  Google Scholar 

  5. Chen, G.D., Wang, C.Y., et al.: A survey on storytelling with robots. In: Proceedings of International Conference on Technologies for E-Learning and Digital Entertainment, pp. 450–456. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-23456-9_81

  6. Costa, S., Brunete, A., Bae, B.C., Mavridis, N.: Emotional storytelling using virtual and robotic agents. Int. J. Humanoid Rob. 15(03), 1850006 (2018)

    Article  Google Scholar 

  7. Elliott, C., Brzezinski, J., Sheth, S., Salvatoriello, R.: Story-morphing in the affective reasoning paradigm: generating stories semi-automatically for use with “emotionally intelligent” multimedia agents. In: Proceedings of the 2nd International Conference on Autonomous Agents (Agents), pp. 181–188 (1998)

    Google Scholar 

  8. Fink, J.: Anthropomorphism and human likeness in the design of robots and human-robot interaction. In: Ge, S.S., Khatib, O., Cabibihan, J.-J., Simmons, R., Williams, M.-A. (eds.) ICSR 2012. LNCS (LNAI), vol. 7621, pp. 199–208. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34103-8_20

    Chapter  Google Scholar 

  9. Gomez, R.: Meet Haru the unassuming big-eyed robot helping researchers study social robotics. IEEE Spectr. (2020)

    Google Scholar 

  10. Gomez, R., et al.: Exploring affective storytelling with an embodied agent. In: Proceedings of 2021 30th IEEE International Conference on Robot & Human Interactive Communication (RO-MAN), pp. 1249–1255. IEEE (2021)

    Google Scholar 

  11. Hubbard, L.J., Chen, Y., Colunga, E., Kim, P., Yeh, T.: Child-robot interaction to integrate reflective storytelling into creative play. In: Proceedings of Creativity and Cognition, pp. 1–8 (2021)

    Google Scholar 

  12. Nichols, E., Gao, L., Gomez, R.: Collaborative storytelling with large-scale neural language models. In: Motion, Interaction and Games, pp. 1–10 (2020)

    Google Scholar 

  13. Nichols, E., Gao, L., Vasylkiv, Y., Gomez, R.: Collaborative storytelling with social robots. In: Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 1903–1910. IEEE (2021)

    Google Scholar 

  14. Park, H.W., Grover, I., Spaulding, S., Gomez, L., Breazeal, C.: A model-free affective reinforcement learning approach to personalization of an autonomous social robot companion for early literacy education. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 33, pp. 687–694 (2019)

    Google Scholar 

  15. de Pinto, M.G., Polignano, M., Lops, P., Semeraro, G.: Emotions understanding model from spoken language using deep neural networks and mel-frequency cepstral coefficients. In: Proceedings of 2020 IEEE Conference on Evolving and adaptive intelligent Systems (EAIS), pp. 1–5. IEEE (2020)

    Google Scholar 

  16. Shihab, K., Ramadhan, H.: Modeling and evaluating of emotional processes. Editorial Advisory Board, p. 128 (2008)

    Google Scholar 

  17. Steinhaeusser, S.C., Schaper, P., Bediako Akuffo, O., Friedrich, P., Ön, J., Lugrin, B.: Anthropomorphize me! Effects of robot gender on listeners’ perception of the social robot NAO in a storytelling use case. In: Proceedings of Companion of the 2021 ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 529–534 (2021)

    Google Scholar 

  18. Striepe, H., Lugrin, B.: There once was a robot storyteller: measuring the effects of emotion and non-verbal behaviour. In: Proceedings of International Conference on Social Robotics (ICSR), pp. 126–136. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-70022-9_13

  19. Swain, M., Routray, A., Kabisatpathy, P.: Databases, features and classifiers for speech emotion recognition: a review. Int. J. Speech Technol. 21(1), 93–120 (2018). https://doi.org/10.1007/s10772-018-9491-z

    Article  Google Scholar 

  20. https://www.imdb.com/title/tt9201722/

  21. Wang, X., Xu, W., Liang, B., Li, C.: Design and development of teleoperation system for space robot. J. Harbin Inst. Technol. 42(3), 337–342 (2010)

    Google Scholar 

  22. Xu, J., Matta, K., Islam, S., Nürnberger, A.: German speech recognition system using DeepSpeech. In: Proceedings of the 4th International Conference on Natural Language Processing and Information Retrieval (NLPIR), pp. 102–106 (2020)

    Google Scholar 

  23. Xu, J., Broekens, J., Hindriks, K., Neerincx, M.A.: Effects of a robotic storyteller’s moody gestures on storytelling perception. In: 2015 International Conference on Affective Computing and Intelligent Interaction (ACII), pp. 449–455. IEEE (2015)

    Google Scholar 

  24. Zabala, U., Rodriguez, I., Martínez-Otzeta, J.M., Lazkano, E.: Expressing robot personality through talking body language. Appl. Sci. 11(10), 4639 (2021)

    Article  Google Scholar 

  25. Zhou, K., Sisman, B., Liu, R., Li, H.: Emotional voice conversion: theory, databases and ESD. Speech Commun. 137, 1–18 (2022)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Ocean University of China, Qingdao, China

    Hui Wang, Lei Zhang, Chuanxiong Zheng & Guangliang Li

  2. Honda Research Institute Japan Co., Ltd., Wako, Japan

    Randy Gomez & Keisuke Nakamura

Authors
  1. Hui Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chuanxiong Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Randy Gomez
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Keisuke Nakamura
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Guangliang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangliang Li .

Editor information

Editors and Affiliations

  1. University of Florence, Florence, Italy

    Filippo Cavallo

  2. Qatar University, Doha, Qatar

    John-John Cabibihan

  3. University of Florence, Florence, Italy

    Laura Fiorini

  4. University of Florence, Florence, Italy

    Alessandra Sorrentino

  5. Wichita State University, Wichita, KS, USA

    Hongsheng He

  6. Qingdao University, Qingdao, China

    Xiaorui Liu

  7. National Institute of Advanced Industrial Science and Technology, Tsukuba, Japan

    Yoshio Matsumoto

  8. National University of Singapore, Singapore, Singapore

    Shuzhi Sam Ge

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, H., Zhang, L., Zheng, C., Gomez, R., Nakamura, K., Li, G. (2022). Personalized Storytelling with Social Robot Haru. In: Cavallo, F., et al. Social Robotics. ICSR 2022. Lecture Notes in Computer Science(), vol 13818. Springer, Cham. https://doi.org/10.1007/978-3-031-24670-8_39

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-24670-8_39

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-24669-2

  • Online ISBN: 978-3-031-24670-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation