Skip to main content
SpringerLink
Log in

And How Enjoyable? Converting a User Experience Evaluation Questionnaire into a Voice Conversation

  • Conference paper
  • First Online:
Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023) (UCAmI 2023)

Part of the book series: Lecture Notes in Networks and Systems ((LNNS,volume 835))

  • 218 Accesses

Abstract

Voice interfaces have gained popularity, driven by the proliferation of intelligent assistants and the development of natural language processing. Given their progress, they could be used to implement self-reported instruments such as standardized user experience (UX) evaluation questionnaires, particularly in the response capture mechanism. This novel use of voice interfaces would make sense if the evaluations using the voice mechanism did not differ significantly from the evaluations carried out with the traditional written questionnaire. In addition, given different possible implementations of the voice mechanism, it must be analyzed whether the user experience is affected by the format of the instrument used. This paper compares two implementations of the User Experience Questionnaire in which the capture mechanism is implemented by voice. Results show that in both implementations the evaluations retain their validity, and that the use of conversational patterns improves the user experience and increases the quality of the answers, which present fewer inconsistencies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 219.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 279.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Lallemand, C., Gronier, G.:Méthodes de design UX: 30 méthodes fondamentales pour concevoir et évaluer les systèmes interactifs. 2eme édition; Éditions Eyrolles: Paris, France (2018)

    Google Scholar 

  2. Díaz-Oreiro, I., López, G., Quesada, L., Guerrero, L.A.: UX evaluation with standardized questionnaires in ubiquitous computing and ambient intelligence: a systematic literature review. Adv. Human-Computer Interact. 2021, 1–22 (2021)

    Article  Google Scholar 

  3. Miller, P.V.: Alternative question forms for attitude scale questions in telephone interviews. Public Opin. Q. 48(4), 766–778 (1984)

    Article  Google Scholar 

  4. Fabrigar, L.R., Krosnick, J.A.: Attitude measurement and questionnaire design, pp. 42–47. Blackwell Publishers, Blackwell encyclopedia of social psychology (1995)

    Google Scholar 

  5. Krosnick, J.A., Berent, M.K.: Comparisons of party identification and policy preferences: the impact of survey question format. Am. J. Polit. Sci. 37(3), 941–964 (1993)

    Google Scholar 

  6. Yu, J.H., Albaum, G., Swenson, M.: Is a central tendency error inherent in the use of semantic differential scales in different cultures? Int. J. Mark. Res. 45(2), 1–16 (2003)

    Article  Google Scholar 

  7. de Rada, D., Igúzquiza, V.: ¿Influye el diseño de las preguntas en las respuestas de los entrevistados? Res. Revista Española de Sociología 31(1), 9 (2022)

    Google Scholar 

  8. Moore, R.J., Arar, R.: Conversational UX Design: A Practitioner’s Guide to The Natural Conversation Framework. Morgan & Claypool, New York, (2019)

    Google Scholar 

  9. Sacks, H., Schegloff, E.A., Jefferson, G.: A simplest systematics for the organization of turn taking for conversation. In: Studies in the Organization of Conversational Interaction, pp. 7–55. Academic Press (1978)

    Google Scholar 

  10. Quamar, A., Özcan, F., Miller, D., Moore, R., Niehus, R., Kreulen, J.: Conversational BI: an ontology-driven conversation system for business intelligence applications. Proc. VLDB Endow. 13(12), 3369–3381 (2020)

    Article  Google Scholar 

  11. Liao, Q.V., Geyer, W., Muller, M., Khazaen, Y.: Conversational Interfaces for Information Search. In: Fu, W.T., van Oostendorp, H. (eds.) Understanding and Improving Information Search. HIS, pp. 267–287. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-38825-6_13

    Chapter  Google Scholar 

  12. Vepsäläinen, H., Salovaara, A. Paakki, H.: What Would Be the Principles for Successful Trollbot Design? In: ACM CHI Conference workshop (2019)

    Google Scholar 

  13. Liu, Y., Arnold, A., Dupont, G., Kobus, C., Lancelot, F.: Evaluation of conversational agents for aerospace domain. In: Proceedings of the Joint Conference of the Information Retrieval Communities in Europe (CIRCLE 2020). July 6–9 Samatan, Gers, France (2020)

    Google Scholar 

  14. Kuang, E., Jahangirzadeh Soure, E., Fan, M., Zhao, J., Shinohara, K.: Collaboration with Conversational AI Assistants for UX Evaluation: Questions and How to Ask them (Voice vs. Text). In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–15 (2023)

    Google Scholar 

  15. Zarouali, B., Araujo, T., Ohme, J., de Vreese, C.: Comparing Chatbots and online surveys for (longitudinal) data collection: an investigation of response characteristics, data quality, and user evaluation. Commun. Methods Measures, 1–20 (2023)

    Google Scholar 

  16. Beam, E.A.: Social media as a recruitment and data collection tool: Experimental evidence on the relative effectiveness of web surveys and Chatbots. J. Dev. Econ. 162, 103069 (2023)

    Article  Google Scholar 

  17. Soni, H. et al.: Virtual conversational agents versus online forms: Patient experience and preferences for health data collection. Front. Digital Health, 4 954069 (2022)

    Google Scholar 

  18. Sprengholz, P., Betsch, C.: Ok Google: using virtual assistants for data collection in psychological and behavioral research. Behav. Res. Methods, 1-13 (2021).https://doi.org/10.3758/s13428-021-01629-y

  19. Wei, J., et al.: Understanding How to Administer Voice Surveys through Smart Speakers. Proc. ACM Hum. Comput. Interact. 6(CSCW2), 1–32 (2022)

    Article  MathSciNet  Google Scholar 

  20. Maharjan, R., Rohani, D.A., Bækgaard, P., Bardram, J., Doherty, K.: Can we talk? design implications for the questionnaire-driven self-report of health and wellbeing via conversational agent. In: Proceedings of the 3rd Conference on Conversational User Interfaces, pp. 1–11 (2021)

    Google Scholar 

  21. Laugwitz, B., Held, T., Schrepp, M.: Construction and evaluation of a user experience questionnaire. In: Holzinger, A. (ed.) USAB 2008. LNCS, vol. 5298, pp. 63–76. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-89350-9_6

    Chapter  Google Scholar 

  22. Rauschenberger, M., Olschner, S., Cota, M. P., Schrepp, M., Thomaschewski, J.: Measurement of user experience: a spanish language version of the user experience questionnaire (UEQ). In: The 7th Iberian Conference on Information Systems and Technologies (CISTI 2012) June 2012, pp. 1–6. IEEE (2012)

    Google Scholar 

  23. VoiceFlow homepage. https://www.voiceflow.com. Accessed 1 May 2023

  24. Mata-Serrano, J.C., Díaz-Oreiro, I., López, G., Guerrero, L.A.: Comparing Written and Voice Captured Responses of the User Experience Questionnaire (UEQ). In: Rocha, Á., Ferrás, C., Méndez Porras, A., Jimenez Delgado, E. (eds.) ICITS 2022. LNNS, vol. 414, pp. 519–529. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96293-7_43

    Chapter  Google Scholar 

  25. Díaz-Oreiro, I., López, G., Quesada, L., Guerrero, L.A.: Conversational design patterns for a UX evaluation instrument implemented by voice. In: Rocha, Á., Ferrás, C., Méndez Porras, A., Jimenez Delgado, E. (eds.) ICITS 2022. LNNS, vol. 414, pp. 530–540. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-96293-7_44

    Chapter  Google Scholar 

  26. Schrepp, M., Thomaschewski, J.: Design and validation of a framework for the creation of user experience questionnaires. Int. J. Interact. Multimedia Artif. Intell. 5, 7–88 (2019)

    Google Scholar 

  27. Klein, A., Hinderks, A., Schrepp, M., Thomaschewski, J.: Construction of UEQ+ scales for voice quality (2020)

    Google Scholar 

  28. Hassenzahl, M., Burmester, M., Koller, F.: AttrakDiff: Ein Fragebogen zur Messung wahrgenommener hedonischer und pragmatischer Qualität. In: Mensch Computer. 2003: Interaktion in Bewegung, pp. 187–196 (2003)

    Google Scholar 

  29. Minge, M., Riedel, L. meCUE-Ein modularer Fragebogen zur Erfassung des Nutzungserlebens. In: Mensch & Computer, pp. 89–98 (2013)

    Google Scholar 

Download references

Acknowledgments

This research was partially funded by CITIC and ECCI at the University of Costa Rica, grant number 834-C1–013.

Author information

Authors and Affiliations

  1. University of Costa Rica, San Jose, 11501, Costa Rica

    Ignacio Díaz-Oreiro, Gustavo López & Luis A. Guerrero

Authors
  1. Ignacio Díaz-Oreiro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gustavo López
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Luis A. Guerrero
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ignacio Díaz-Oreiro .

Editor information

Editors and Affiliations

  1. MAmI Research Lab, Castilla-La Mancha University, Ciudad Real, Spain

    José Bravo

  2. Anahuac University, Mérida, Yucatán, Mexico

    Gabriel Urzáiz

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Díaz-Oreiro, I., López, G., Guerrero, L.A. (2023). And How Enjoyable? Converting a User Experience Evaluation Questionnaire into a Voice Conversation. In: Bravo, J., Urzáiz, G. (eds) Proceedings of the 15th International Conference on Ubiquitous Computing & Ambient Intelligence (UCAmI 2023). UCAmI 2023. Lecture Notes in Networks and Systems, vol 835. Springer, Cham. https://doi.org/10.1007/978-3-031-48306-6_32

Download citation

Publish with us

Policies and ethics

Search

Navigation