Skip to main content
SpringerLink
Log in

Applying the Social Robot Expectation Gap Evaluation Framework

  • Conference paper
  • First Online:
Human-Computer Interaction (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14013))

Included in the following conference series:

  • 884 Accesses

Abstract

Expectations shape our experience with the world, including our interaction with technology. There is a mismatch between what humans expect of social robots and what they are actually capable of. Expectations are dynamic and can change over time. We have previously developed a framework for studying these expectations over time in human-robot interaction (HRI). In this work, we applied the social robot expectation gap evaluation framework in an HRI scenario from a UX evaluation perspective, by analyzing a subset of data collected from a larger experiment. The framework is based on three factors of expectation: affect, cognitive processing, as well as behavior and performance. Four UX goals related to a human-robot interaction scenario were evaluated. Results show that expectations change over time with an overall improved UX in the second interaction. Moreover, even though some UX goals were partly fulfilled, there are severe issues with the conversation between the user and the robot, ranging from the quality of the interaction to the users’ utterances not being recognized by the robot. This work takes the initial steps towards disentangling how expectations work and change over time in HRI. Future work includes expanding the metrics to study expectations and to further validate the framework.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. OpenAI. https://openai.com/

  2. Alač, M.: Social robots: things or agents? AI Soc. 31(4), 519–535 (2016)

    Article  Google Scholar 

  3. Alenljung, B., Lindblom, J., Andreasson, R., Ziemke, T.: User experience in social human-robot interaction. In: Rapid Automation: Concepts, Methodologies, Tools, and Applications, pp. 1468–1490. IGI Global (2019)

    Google Scholar 

  4. Billing, E., Rosén, J., Lamb, M.: Language models for human-robot interaction. In: Companion of the 2023 ACM/IEEE International Conference on Human-Robot Interaction (HRI ’23 Companion), 13–16 March 2023, Stockholm, Sweden. ACM, New York, NY, USA (2023). https://doi.org/10.1145/3568294.3580040

  5. Breazeal, C., Dautenhahn, K., Kanda, T.: Social robotics. In: Siciliano, B., Khatib, O. (eds.) Springer Handbook of Robotics, pp. 1935–1972. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-32552-1_72

  6. Dautenhahn, K.: Some brief thoughts on the past and future of human-robot interaction. ACM Trans. Hum.-Robot Interact. (THRI) 7(1), 4 (2018). https://doi.org/10.1145/3209769, https://dl.acm.org/citation.cfm?id=3209769

  7. Edwards, A., Edwards, C., Westerman, D., Spence, P.: Initial expectations, interactions, and beyond with social robots. Comput. Hum. Behav. 90, 308–314 (2019)

    Article  Google Scholar 

  8. Hartson, H., Pyla, P.: The UX Book. Morgan Kaufmann (2018)

    Google Scholar 

  9. Hassenzahl, M., Tractinsky, N.: User experience - a research agenda. Behav. Inf. Technol. 25(2), 91–97 (2006)

    Article  Google Scholar 

  10. Horstmann, A., Krämer, N.: Great expectations? Relation of previous experiences with social robots in real life or in the media and expectancies based on qualitative and quantitative assessment. Front. Psychol. 10, 939 (2019)

    Article  Google Scholar 

  11. Horstmann, A., Krämer, N.: When a robot violates expectations. In: Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pp. 254–256 (2020)

    Google Scholar 

  12. Jokinen, K., Wilcock, G.: Expectations and first experience with a social robot. In: Proceedings of the 5th International Conference on Human Agent Interaction, pp. 511–515 (2017)

    Google Scholar 

  13. Kaasinen, E., Kymäläinen, T., Niemelä, M., Olsson, T., Kanerva, M., Ikonen, V.: A user-centric view of intelligent environments. Computers 2(1), 1–33 (2013)

    Article  Google Scholar 

  14. Kwon, M., Jung, M., Knepper, R.: Human expectations of social robots. In: 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 463–464. IEEE (2016)

    Google Scholar 

  15. Lindblom, J., Alenljung, B., Billing, E.: Evaluating the user experience of human-robot interaction. In: Lindblom, J., Alenljung, B., Billing, E. (eds.) Human-Robot Interaction, pp. 231–256, vol. 12. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-42307-0_9

  16. Lohse, M.: The role of expectations in HRI. New Front. Hum.-Robot Interact. 35–56 (2009)

    Google Scholar 

  17. Mahdi, H., Akgun, S.A., Saleh, S., Dautenhahn, K.: A survey on the design and evolution of social robots-past, present and future. Robot. Auton. Syst. 104193 (2022)

    Google Scholar 

  18. Nomura, T., Kanda, T., Suzuki, T., Kato, K.: Psychology in human-robot communication. In: RO-MAN 2004. 13th IEEE International Workshop on Robot and Human Interactive Communication (IEEE Catalog No. 04TH8759), pp. 35–40. IEEE (2004)

    Google Scholar 

  19. Nomura, T., Suzuki, T., Kanda, T., Kato, K.: Measurement of anxiety toward robots. In: ROMAN 2006-The 15th IEEE International Symposium on Robot and Human Interactive Communication, pp. 372–377. IEEE (2006)

    Google Scholar 

  20. Olson, J., Roese, N., Zanna, M.: Expectancies, pp. 211–238. Guilford Press (1996)

    Google Scholar 

  21. Paetzel, M., Perugia, G., Castellano, G.: The persistence of first impressions: the effect of repeated interactions on the perception of a social robot. In: Proceedings of the 2020 ACM/IEEE International Conference on Human-Robot Interaction, pp. 73–82 (2020)

    Google Scholar 

  22. Patton, M.Q.: Qualitative Research & Evaluation Methods: Integrating Theory and Practice. Sage Publications, Thousand Oaks (2014)

    Google Scholar 

  23. Rosén, J.: Expectations in human-robot interaction. In: Ayaz, H., Asgher, U., Paletta, L. (eds.) AHFE 2021. LNNS, vol. 259, pp. 98–105. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-80285-1_12

    Chapter  Google Scholar 

  24. Rosén, J., Lindblom, J., Billing, E.: The social robot expectation gap evaluation framework. In: Kurosu, M. (eds.) HCII 2022. LNCS, vol. 13303, pp. 590–610. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-05409-9_43

  25. Roto, V., Law, E., Vermeeren, A., Hoonhout, J.: User experience white paper - bringing clarity to the concept of user experience. In Dagstuhl Seminar on Demarcating User Experience (2011)

    Google Scholar 

  26. Sandoval, E.B., Mubin, O., Obaid, M.: Human robot interaction and fiction: a contradiction. In: Beetz, M., Johnston, B., Williams, M.-A. (eds.) ICSR 2014. LNCS (LNAI), vol. 8755, pp. 54–63. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11973-1_6

    Chapter  Google Scholar 

  27. Serholt, S., Barendregt, W.: Robots tutoring children: longitudinal evaluation of social engagement in child-robot interaction. In: Proceedings of the 9th Nordic Conference on Human-Computer Interaction, pp. 1–10 (2016)

    Google Scholar 

  28. SoftBank Robotics: https://www.softbankrobotics.com (2018). Accessed 05 Jan 2018

  29. Stephanidis, C., et al.: Seven HCI grand challenges. Int. J. Hum.-Comput. Interact. 7318 (2019)

    Google Scholar 

  30. Woosnam, K.M., et al.: The inclusion of other in the self (iOS) scale. Ann. Tour. Res. 37(3), 857–860 (2010)

    Article  Google Scholar 

Download references

Acknowledgements

We would like to give a big thanks to the users that participated in this study, including Erik Lagerstedt who agreed to be depicted in Fig. 2.

This study was submitted for ethical review to the Swedish Ethical Review Authority (#2022-02582-01, Linköping) and was found to not require ethical review under Swedish legislation (2003:615). There were no physical or mental health risks to the users, and they were informed of their tasks prior to receiving an informed consent form. All data have been de-identified during collection. No sensitive personal information was collected. Video recordings are stored locally on a computer that is password protected. These recordings are only available for the researchers that analyzed the data and will be deleted after publication.

Author information

Authors and Affiliations

  1. Interaction Lab, University of Skövde, Box 408, 541 28, Skövde, Sweden

    Julia Rosén, Erik Billing & Jessica Lindblom

  2. Department of Information Technology, Uppsala University, Box 337, 751 05, Uppsala, Sweden

    Jessica Lindblom

Authors
  1. Julia Rosén
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Erik Billing
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jessica Lindblom
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jessica Lindblom .

Editor information

Editors and Affiliations

  1. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  2. Hosei University, Tokyo, Japan

    Ayako Hashizume

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rosén, J., Billing, E., Lindblom, J. (2023). Applying the Social Robot Expectation Gap Evaluation Framework. In: Kurosu, M., Hashizume, A. (eds) Human-Computer Interaction. HCII 2023. Lecture Notes in Computer Science, vol 14013. Springer, Cham. https://doi.org/10.1007/978-3-031-35602-5_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-35602-5_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-35601-8

  • Online ISBN: 978-3-031-35602-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation