Abstract
In human-human interaction, the participants’ multimodal behaviour has impact on the interaction as a whole, and similarly in spoken human-robot interactions, the interlocutors’ multimodal signals seem to correlate with the user’s experience and impressions of the interaction. We explored in more detail how some aspects of multimodal behaviour (gazing, facial expressions, body posture) can predict the user’s evaluation of the robot’s behaviour (Responsiveness, Expressiveness, Interface, Usability, Overall impression). The results indicate that the user’s assessment concerning the evaluation categories Interface and Usability, and to some extent the categories Expressiveness and Overall correlate with their behaviour in a statistically significant manner. The work contributes to our understanding of how the interlocutors’ engagement and active participation relate to their assessment of the success of communication, and points towards automating evaluation of human-robot interactions.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Anastasiou, D., Jokinen, K., Wilcock, G.: Evaluation of WikiTalk – user studies of human-robot interaction. In: Kurosu, M. (ed.) HCII/HCI 2013, Part IV. LNCS, vol. 8007, pp. 32–42. Springer, Heidelberg (2013)
Bavelas, J.B., Black, A., Lemery, C.R., Mullett, J.: I show how you feel: motor mimicry as a communicative act. J. Pers. Soc. Psychol. 50(2), 322–329 (1986)
Beringer, N., Kartal, U., Louka, K., Schiel, F., Türk, U.: PROMISE: a procedure for multimodal interactive system evaluation. In: Proceedings of the Workshop Multimodal Resources and Multimodal Systems Evaluation, Las Palmas (2002)
Breazeal, C., Scassellati, B.: A context-dependent attention system for a social robot. In: Proceedings of the 16th International Joint Conference in Artificial Intelligence (IJCAI 1999), Stockholm, pp. 1146–1151 (1999)
Campbell, N., Scherer, S.: Comparing measures of synchrony and alignment in dialogue speech timing with respect to turn-taking activity. In: Proceedings of Interspeech 2010, Makuhari (2010)
Csapo, A., Gilmartin, E., Grizou, J., Han, J., Meena, R., Anastasiou, D., Jokinen, K., Wilcock, G.: Multimodal conversational interaction with a humanoid robot. In: Proceedings of 3rd IEEE International Conference on Cognitive Infocommunications (CogInfoCom 2012), Kosice, pp. 667–672 (2012)
Edlund, J., Heldner, M., Hirschberg, J.: Pause and gap length in face-to-face interaction. In: Proceedings of Interspeech 2009, Brighton (2009)
Endsley, M.R.: Theoretical underpinnings of situation awareness: a critical review. In: Endsley, M.R., Garland, D.J. (eds.) Situation Awareness Analysis and Measurement. LEA, Mahwah (2000)
Goodwin, C.: Action and embodiment within situated human interaction. J. Pragmat. 32, 1489–1522 (2000)
Han, J., Campbell, N., Jokinen, K., Wilcock, G.: Investigating the use of non-verbal cues in human-robot interaction with a Nao robot. In: Proceedings of 3rd IEEE International Conference on Cognitive Infocommunications (CogInfoCom 2012), Kosice, pp. 679–683 (2012)
Ishii, R., Nakano, Y., Nishida, T.: Gaze awareness in conversational agents: Estimating a user’s conversational engagement from eye gaze. ACM Trans. Interact. Intell. Syst. 3(2), 11:1–11:25 (2013)
Jokinen, K., McTear, M.: Spoken Dialogue Systems. Morgan and Claypool, San Rafael (2009)
Jokinen, K., Wilcock, G.: User experience in human-robot interactions. In: Schatz, R., Hossfeld, T. (eds.) Proceedings of the Fourth Workshop on Perceptual Quality of Systems (PQS 2013), FTW-TECHREPORT-128, FTW, Vienna, Austria (2013)
Levitan, R., Gravano, A., Hirschberg, J.: Entrainment in speech preceding backchannels. In: Proceedings of ACL 2011, pp. 113–117 (2011)
Levitski, A., Radun, J., Jokinen, K.: Visual interaction and conversational activity. In: Proceedings of the 4th Workshop on Eye Gaze in Intelligent Human Machine Interaction: Eye Gaze and Multimodality, at the 14th ACM International Conference on Multimodal Interaction (ICMI 2012), Santa Monica, California, USA (2012)
Nakano, Y., Reinstein, G., Stocky, T., Cassell, J.: Towards a model of face-to-face grounding. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, vol. 1, pp. 553–561 (2003)
Steinfeld, A., Fong, T., Kaber, D., Scholtz, J., Schultz, A., Goodrich, M.: Common metrics for human-robot interaction. In: 2006 Human-Robot Interaction Conference (2006)
Traum, D., Robinson, S., Stephan, J.: Evaluation of multi-party virtual reality dialogue interaction. In: Proceedings of Fourth International Conference on Language Resources and Evaluation (LREC 2004), Lisbon, pp. 1699–1702 (2004)
Walker, M., Litman, D., Kamm, C., Abella, A.: PARADISE: a framework for evaluating spoken dialogue agents. In: Proceedings of the 35th Annual Meeting of the Association for Computational Linguistics, Madrid (1997)
Wilcock, G.: WikiTalk: a spoken Wikipedia-based open-domain knowledge access system. In: Proceedings of the COLING 2012 Workshop on Question Answering in Complex Domains, Mumbai, India, pp. 57–69 (2012)
Witten, I., Frank, E., Hall, M.: Data Mining: Practical Machine Learning Tools and Techniques, 3rd edn. Morgan Kaufmann, San Francisco (2011)
Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., Sloetjes, H.: ELAN: a professional framework for multimodality research. In: Proceedings of the Fifth International Conference on Language Resources and Evaluation (LREC 2006), pp. 1556–1559 (2006)
Acknowledgements
We would like to thank all the participants as well as the annotators of the video files.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Jokinen, K., Wilcock, G. (2015). Modelling User Experience in Human-Robot Interactions. In: Böck, R., Bonin, F., Campbell, N., Poppe, R. (eds) Multimodal Analyses enabling Artificial Agents in Human-Machine Interaction. MA3HMI 2014. Lecture Notes in Computer Science(), vol 8757. Springer, Cham. https://doi.org/10.1007/978-3-319-15557-9_5
Download citation
DOI: https://doi.org/10.1007/978-3-319-15557-9_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-15556-2
Online ISBN: 978-3-319-15557-9
eBook Packages: Computer ScienceComputer Science (R0)