skip to main content
10.1145/3180308.3180362acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
poster

How Personal Experience and Technical Knowledge Affect Using Conversational Agents

Published: 05 March 2018 Publication History

Abstract

Conversational agents (CA) use dialogues to interact with users so as to offer an experience of naturalistic interaction. However, due to the low transparency and poor explanability of mechanism inside CA, individual's understanding of CA's capabilities may affect how the individual interacts with CA and the sustainability of CA use. To examine how users' understanding affect perceptions and experiences of using CA, we conducted a laboratory study asking 41 participants performed a set of tasks using Apple Siri. We independently manipulated two factors: (1) personal experience of using CA, and (2) technical knowledge about CA's system model. We conducted mixed-method analyses of post-task usability measures and interviews, and confirmed that use experience and technical knowledge affects perceived usability and mental models differently.

References

[1]
Benjamin R. Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. "What Can I Help You with?": Infrequent Users' Experiences of Intelligent Personal Assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI '17). ACM, New York, NY, USA, Article 43, 12 pages.
[2]
Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf Between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 5286--5297.
[3]
Joe Tullio, Anind K. Dey, Jason Chalecki, and James Fogarty. 2007. How It Works: A Field Study of Non-technical Users Interacting with an Intelligent System. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, New York, NY, USA, 31--40.

Cited By

View all
  • (2024)Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent UsersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642542(1-18)Online publication date: 11-May-2024
  • (2024)Towards a general user model to develop intelligent user interfacesMultimedia Tools and Applications10.1007/s11042-024-18240-w83:26(67501-67534)Online publication date: 25-Jan-2024
  • (2023)Using the SOCIO Chatbot for UML Modelling: A Family of ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2022.315072049:1(364-383)Online publication date: 1-Jan-2023
  • Show More Cited By

Index Terms

  1. How Personal Experience and Technical Knowledge Affect Using Conversational Agents

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    IUI '18 Companion: Companion Proceedings of the 23rd International Conference on Intelligent User Interfaces
    March 2018
    141 pages
    ISBN:9781450355711
    DOI:10.1145/3180308
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 05 March 2018

    Check for updates

    Author Tags

    1. Conversational agents
    2. explainable intelligent user interfaces
    3. mental models

    Qualifiers

    • Poster
    • Research
    • Refereed limited

    Conference

    IUI'18
    Sponsor:

    Acceptance Rates

    IUI '18 Companion Paper Acceptance Rate 63 of 127 submissions, 50%;
    Overall Acceptance Rate 746 of 2,811 submissions, 27%

    Upcoming Conference

    IUI '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)26
    • Downloads (Last 6 weeks)3
    Reflects downloads up to 14 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Listening to the Voices: Describing Ethical Caveats of Conversational User Interfaces According to Experts and Frequent UsersProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642542(1-18)Online publication date: 11-May-2024
    • (2024)Towards a general user model to develop intelligent user interfacesMultimedia Tools and Applications10.1007/s11042-024-18240-w83:26(67501-67534)Online publication date: 25-Jan-2024
    • (2023)Using the SOCIO Chatbot for UML Modelling: A Family of ExperimentsIEEE Transactions on Software Engineering10.1109/TSE.2022.315072049:1(364-383)Online publication date: 1-Jan-2023
    • (2022)Software-Based Dialogue Systems: Survey, Taxonomy, and ChallengesACM Computing Surveys10.1145/352745055:5(1-42)Online publication date: 3-Dec-2022
    • (2022)Exploring the Opinions of Experts in Conversational Design: A Study on Users’ Mental Models of Voice AssistantsHuman-Computer Interaction. User Experience and Behavior10.1007/978-3-031-05412-9_34(494-514)Online publication date: 26-Jun-2022
    • (2021)Teachers’ attitudes towards chatbots in education: a technology acceptance model approach considering the effect of social language, bot proactiveness, and users’ characteristicsEducational Studies10.1080/03055698.2020.185042649:2(295-313)Online publication date: 4-Feb-2021
    • (2021)Diabetes and conversational agents: the AIDA project case studyDiscover Artificial Intelligence10.1007/s44163-021-00005-11:1Online publication date: 22-Sep-2021
    • (2020)Collaborative ModellingProceedings of the 24th International Conference on Evaluation and Assessment in Software Engineering10.1145/3383219.3383246(260-269)Online publication date: 15-Apr-2020
    • (2020)Evaluation Techniques for Chatbot Usability: A Systematic Mapping StudyInternational Journal of Software Engineering and Knowledge Engineering10.1142/S021819401940016329:11n12(1673-1702)Online publication date: 11-Feb-2020
    • (2020)QUESTO: Interactive Construction of Objective Functions for Classification TasksComputer Graphics Forum10.1111/cgf.1397039:3(153-165)Online publication date: 18-Jul-2020
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media