Abstract
Assessment of usability and user experience of spoken dialog services is a rather complex task, which remains difficult to achieve with real end-users. In this work a three-fold evaluation approach is introduced, which supports reliable assessment of usability and user experience. The approach combines interaction log data based assessment (at dialog, task and node level) with an optimized questionnaire-based end-users evaluation and a controlled stress test performed by an IVR system. The 3-fold evaluation approach was used for the assessment of usability and user experience of the pilot deployment of a voice banking systems. The proposed assessment approach provides sufficient evidence for the business informed decision-making with respect to perceived user quality of the interaction and offered services and allows for investigation of potential improvement areas.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Harris, D.: IVR Systems – Buyer’s Guide (2017). http://www.softwareadvice.com/call-center/ivr-system-comparison/
Mporas, I., Kocsis, O., Ganchev, T., Fakotakis, N.: Robust speech interaction in motorcycle environment. Expert Syst. Appl. 37(3), 1827–1835 (2010)
Diewald, S., Moller, A., Roalter, L., Kranz, M.: Mobile device integration and interaction in the automotive domain. In: Proceedings of AutomotiveUI 2011 (2012)
Strauss, P.M., Minker, W.: Proactive Spoken Dialogue Interaction in Multi-Party Environments. Springer Science + Business Media, New York (2010)
Kocsis, O., Kladis, B.: Entrance manager application: alternative ways for event handling at the front door of a smart house. In: Janse, M.D. (ed.) Amigo Project Workshop Proceedings, pp. 47–48 (2008)
Almeida, N., Silva, S., Teixeira, A.: Design and development of speech interaction: a methodology. In: Kurosu, M. (ed.) HCI 2014. LNCS, vol. 8511, pp. 370–381. Springer, Cham (2014). doi:10.1007/978-3-319-07230-2_36
Moller, S.: Evaluating interaction with spoken dialogue telephone services. Chapter in Recent Trends in Discourse and Dialogue 39, 69–100 (2008)
Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of system usability scale. Int. J. Human-Comput. Inter. 24(6), 574–594 (2008) doi:10.1080/10447310802205776
Turunen, M., Hakulinen, J., Melto, A., Heimonen, T., Laivo, T., Hella, J.: SUXES – User experience evaluation method for spoken and multimodal interaction. In: Proceedings of INTERSPEECH 2009, pp. 2567–2570 (2009)
Jurcicek, F., Keizer, S., Gasic, M., Mairesse, F., Thomson, B., Yu, K., Young, S.: Real user evaluation of spoken dialogue systems using Amazon Mechanic Turk. In: Proceedings of INTERSPEECH 2011 (2011)
Metze, F., Englert, R., Bub, U., Burkhardt, F., Stegmann, J.: Getting closer: tailored human-computer speech dialog. Univ. Access Inf. Soc. 8, 97–108 (2009), doi:10.1007/s10209-008-0133-0
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Kocsis, O., Kladis, B., Tsopanoglou, A., Fakotakis, N. (2017). Assessing Spoken Dialog Services from the End-User Perspective: Usability and Experience. In: Karpov, A., Potapova, R., Mporas, I. (eds) Speech and Computer. SPECOM 2017. Lecture Notes in Computer Science(), vol 10458. Springer, Cham. https://doi.org/10.1007/978-3-319-66429-3_15
Download citation
DOI: https://doi.org/10.1007/978-3-319-66429-3_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66428-6
Online ISBN: 978-3-319-66429-3
eBook Packages: Computer ScienceComputer Science (R0)