Skip to main content
SpringerLink
Log in

Measuring User Experience in Chatbots: An Approach to Interpersonal Communication Competence

  • Conference paper
  • First Online:
Book cover Internet Science (INSCI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11551))

Included in the following conference series:

Abstract

The increasing usage of chatbots is fundamentally changing the way people interact with new technology. Instead of clicking buttons to functionally navigate on a web page, people can access content and services by the use of natural language in interaction with an artificial agent (e.g., chatbot). This change toward human–chatbot interaction is typically manifested through a social and natural conversational style. This shift of how to interact with data and services has major repercussions for how to explore and measure conversational user experience with chatbots. Hence, in this paper we suggest the importance of measuring the social aspects in human–chatbot interaction through a focus on interpersonal communicational competence (ICC). We build our suggested framework on previous knowledge about communicational competence in successful human–human interaction. Based on this we have developed factors that might be of importance for chatbots’ ICC and how these can be measured.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 49.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 64.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Dale, R.: The return of the chatbots. Nat. Lang. Eng. 22(5), 811–817 (2016). https://doi.org/10.1017/S1351324916000243

    Article  Google Scholar 

  2. Brandtzaeg, P.B., Følstad, A.: Chatbots: changing user needs and motivations. Interactions 25(5), 38–43 (2018)

    Article  Google Scholar 

  3. Monk, A.: User-centred design. In: Sloane, A., van Rijn, F. (eds.) Home Informatics and Telematics. ITIFIP, vol. 45, pp. 181–190. Springer, Boston (2000). https://doi.org/10.1007/978-0-387-35511-5_14

    Chapter  Google Scholar 

  4. Brandtzæg, P.B., Følstad, A., Heim, J.: Enjoyment: lessons from Karasek. In: Blythe, M.A., Overbeeke, K., Monk, A.F., Wright, P.C. (eds.) Funology. Human-Computer Interaction Series, vol. 3, pp. 331–341. Springer, Dordrecht (2018). https://doi.org/10.1007/1-4020-2967-5_6

    Chapter  Google Scholar 

  5. Nass, C., Steuer, J., Tauber, E.R.: Computers are social actors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Boston, Massachusetts, USA, pp. 72–78 (1994). https://doi.org/10.1145/191666.191703

  6. Bargas-Avila, J.A., Hornbæk, K.: Old wine in new bottles or novel challenges: a critical analysis of empirical studies of user experience. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Vancouver, BC, Canada, pp. 2689–2698 (2011). https://doi.org/10.1145/1978942.1979336

  7. Brooke, J.: SUS-A quick and dirty usability scale. In: Usability Evaluation in Industry, vol. 189, no. 194, pp. 4–7 (1996)

    Google Scholar 

  8. Ho, C.-C., MacDorman, K.F.: Revisiting the uncanny valley theory: developing and validating an alternative to the Godspeed indices. Comput. Hum. Behav. 26(6), 1508–1518 (2010). https://doi.org/10.1016/j.chb.2010.05.015

    Article  Google Scholar 

  9. Biocca, F., Harms, C., Burgoon, J.K.: Toward a more robust theory and measure of social presence: review and suggested criteria. Presence: Teleoper. Virtual Environ. 12(5), 456–480 (2003). https://doi.org/10.1162/105474603322761270

    Article  Google Scholar 

  10. Spitzberg, B.H., Hecht, M.L.: A component model of relational competence. Hum. Commun. Res. 10(4), 575–599 (1984). https://doi.org/10.1111/j.1468-2958.1984.tb00033.x

    Article  Google Scholar 

  11. Rubin, R.B., Martin, M.M.: Development of a measure of interpersonal communication competence. Commun. Res. Rep. 11(1), 33–44 (1994). https://doi.org/10.1080/08824099409359938

    Article  Google Scholar 

  12. Anders, S.L., Tucker, J.S.: Adult attachment style, interpersonal communication competence, and social support. Pers. Relat. 7(4), 379–389 (2000). https://doi.org/10.1111/j.1475-6811.2000.tb00023.x

    Article  Google Scholar 

  13. De Graaf, M.M., Allouch, S.B.: Exploring influencing variables for the acceptance of social robots. Robot. Auton. Syst. 61(12), 1476–1486 (2013). https://doi.org/10.1016/j.robot.2013.07.007

    Article  Google Scholar 

  14. De Graaf, M., Ben Allouch, S., Van Dijk, J.: Why do they refuse to use my robot?: Reasons for non-use derived from a long-term home study. In: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, Vienna, Austria, pp. 224–233 (2017). https://doi.org/10.1145/2909824.3020236

  15. Medhi Thies, I., Menon, N., Magapu, S., Subramony, M., O’Neill, J.: How do you want your chatbot? An exploratory Wizard-of-Oz study with young, urban indians. In: Bernhaupt, R., Dalvi, G., Joshi, A., Balkrishan, D.K., O’Neill, J., Winckler, M. (eds.) INTERACT 2017. LNCS, vol. 10513, pp. 441–459. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67744-6_28

    Chapter  Google Scholar 

  16. Fitzpatrick, K.K., Darcy, A., Vierhile, M.: Delivering cognitive behavior therapy to young adults with symptoms of depression and anxiety using a fully automated conversational agent (Woebot): a randomized controlled trial. JMIR Mental Health 4(2), e19 (2017). https://doi.org/10.2196/mental.7785

    Article  Google Scholar 

  17. Kim, J., et al.: Can a machine tend to teenagers’ emotional needs?: A study with conversational agents. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal QC, Canada, pp. LBW018 (2018). https://doi.org/10.1145/3170427.3188548

  18. Marrinan, F.: Qualitative investigations into a virtual CBT therapist: relational features, client experiences and implications for counselling psychology practice. Doctoral thesis, University of Surrey (2018)

    Google Scholar 

  19. Luger, E., Sellen, A.: Like having a really bad PA: the gulf between user expectation and experience of conversational agents. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, pp. 5286–5297 (2016). https://doi.org/10.1145/2858036.2858288

  20. Curry, A.C., Rieser, V.: # MeToo Alexa: how conversational systems respond to sexual harassment. In: Proceedings of the Second ACL Workshop on Ethics in Natural Language Processing, New Orleans, Louisiana, pp. 7–14 (2018)

    Google Scholar 

  21. Di Gaetano, S., Diliberto, P.: Chatbots and conversational interfaces: three domains of use. In: Fifth International Workshop on Cultures of Participation in the Digital Age, Castiglione della Pescaia, Italy, vol. 2101, pp. 62–70 (2018)

    Google Scholar 

  22. Spitzberg, B.H.: Preliminary development of a model and measure of computer-mediated communication (CMC) competence. J. Comput.-Mediat. Commun. 11(2), 629–666 (2006). https://doi.org/10.1111/j.1083-6101.2006.00030.x

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. SINTEF, Oslo, Norway

    Marita Skjuve

  2. Department of Media and Communications, University of Oslo, Oslo, Norway

    Marita Skjuve & Petter Bae Brandzaeg

Authors
  1. Marita Skjuve
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Petter Bae Brandzaeg
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Marita Skjuve .

Editor information

Editors and Affiliations

  1. St. Petersburg State University, St. Petersburg, Russia

    Svetlana S. Bodrunova

  2. National Research University Higher School of Economics, St. Petersburg, Russia

    Olessia Koltsova

  3. SINTEF, Trondheim, Norway

    Asbjørn Følstad

  4. Inria, Le Chesnay, France

    Harry Halpin

  5. National Research University Higher School of Economics, Moscow, Russia

    Polina Kolozaridi

  6. National Research University Higher School of Economics, Moscow, Russia

    Leonid Yuldashev

  7. St. Petersburg State University, St. Petersburg, Russia

    Anna Smoliarova

  8. TU München, Munich, Germany

    Heiko Niedermayer

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Skjuve, M., Brandzaeg, P.B. (2019). Measuring User Experience in Chatbots: An Approach to Interpersonal Communication Competence. In: Bodrunova, S., et al. Internet Science. INSCI 2018. Lecture Notes in Computer Science(), vol 11551. Springer, Cham. https://doi.org/10.1007/978-3-030-17705-8_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-17705-8_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-17704-1

  • Online ISBN: 978-3-030-17705-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation