Work in Progress

Expectation vs Reality in Users’ Willingness to Delegate to Digital Assistants

Authors:

Ekaterina Svikhnushina,

Marcel Schellenberg,

Anna K Niedbala,

Jeremy N MilesAuthors Info & Claims

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

Article No.: 107, Pages 1 - 7

https://doi.org/10.1145/3544549.3585763

Published: 19 April 2023 Publication History

Abstract

We report results of a survey with 2500 US-based users of popular digital assistants (DAs) to understand and prioritize the factors that drive consumer adoption. Using structural equation modelling, we investigated the relationship between respondents’ behavioural intentions to delegate tasks to their DAs and two predictor layers: DA (users’ attitudes and familiarity with them) and task factors (need for control/transparency, subjectivity, risk, self-efficacy, and frequency), mediated by a values layer (trust, perceived ease of use and usefulness). Perceived usefulness and trust were strong direct predictors of willingness to delegate. Both DA-related factors had indirect effects, with a surprising negative influence of familiarity on both trust and usefulness. Our novel findings of task-related effects on the value factors may explain this disparity. These results imply a mismatch between users’ expectations for DAs and their actual experiences. We interpret these findings in light of related work and derive implications for practitioners.

Supplementary Material

Supplemental Materials (3544549.3585763-supplemental-materials.zip)

Download
526.98 KB

MP4 File (3544549.3585763-talk-video.mp4)

Pre-recorded Video Presentation

Download
28.38 MB

References

[1]

Praveen Aggarwal and Tridib Mazumdar. 2008. Decision delegation: A conceptualization and empirical investigation. Psychology & Marketing 25, 1 (2008), 71–93. https://doi.org/10.1002/mar.20201

[2]

Icek Ajzen 1991. The theory of planned behavior. Organizational behavior and human decision processes 50, 2 (1991), 179–211.

[3]

Tawfiq Ammari, Jofish Kaye, Janice Y. Tsai, and Frank Bentley. 2019. Music, Search, and IoT: How People (Really) Use Voice Assistants. ACM Trans. Comput.-Hum. Interact. 26, 3, Article 17 (apr 2019), 28 pages. https://doi.org/10.1145/3311956

Digital Library

[4]

Muhammad Ashfaq, Jiang Yun, Shubin Yu, and Sandra Maria Correia Loureiro. 2020. I, Chatbot: Modeling the determinants of users’ satisfaction and continuance intention of AI-powered service agents. Telematics and Informatics 54 (2020), 101473. https://doi.org/10.1016/j.tele.2020.101473

[5]

Daniel Belanche, Luis V Casaló, and Carlos Flavián. 2019. Artificial Intelligence in FinTech: understanding robo-advisors adoption among customers. Industrial Management & Data Systems (2019).

[6]

Izak Benbasat and Weiquan Wang. 2005. Trust in and adoption of online recommendation agents. Journal of the association for information systems 6, 3 (2005), 4.

[7]

Anna Bouwer. 2022. Under Which Conditions Are Humans Motivated to Delegate Tasks to AI? A Taxonomy on the Human Emotional State Driving the Motivation for AI Delegation. In Marketing and Smart Technologies, José Luís Reis, Eduardo Parra López, Luiz Moutinho, and José Paulo Marques dos Santos (Eds.). Springer Nature Singapore, Singapore, 37–53.

[8]

Jason W. Burton, Mari-Klara Stein, and Tina Blegind Jensen. 2020. A systematic review of algorithm aversion in augmented decision making. Journal of Behavioral Decision Making 33, 2 (2020), 220–239. https://doi.org/10.1002/bdm.2155

[9]

Cristiano Castelfranchi and Rino Falcone. 1998. Towards a theory of delegation for agent-based systems. Robotics and Autonomous Systems 24, 3 (1998), 141–157. https://doi.org/10.1016/S0921-8890(98)00028-1 Multi-Agent Rationality.

[10]

Noah Castelo, Maarten W Bos, and Donald R Lehmann. 2019. Task-dependent algorithm aversion. Journal of Marketing Research 56, 5 (2019), 809–825.

[11]

Derwin K. C. Chan, Andreas Ivarsson, Andreas Stenling, Sophie X. Yang, Nikos L. D. Chatzisarantis, and Martin S. Hagger. 2015. Response-order effects in survey methods: A randomized controlled crossover study in the context of sport injury prevention.Journal of Sport & Exercise Psychology 37 (2015), 666–673. https://doi.org/10.1123/jsep.2015-0045

[12]

Chih-Yang Chao, Tsai-Chu Chang, Hui-Chun Wu, Yong-Shun Lin, and Po-Chen Chen. 2016. The interrelationship between intelligent agents’ characteristics and users’ intention in a search engine by making beliefs and perceived risks mediators. Computers in Human Behavior 64 (2016), 117–125.

Digital Library

[13]

Chih-Yang Chao, Tsai-Chu Chang, Hui-Chun Wu, Yong-Shun Lin, and Po-Chen Chen. 2016. The interrelationship between intelligent agents’ characteristics and users’ intention in a search engine by making beliefs and perceived risks mediators. Computers in Human Behavior 64 (2016), 117–125. https://doi.org/10.1016/j.chb.2016.06.031

Digital Library

[14]

Benjamin R. Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. "What Can i Help You with?": Infrequent Users’ Experiences of Intelligent Personal Assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services (Vienna, Austria) (MobileHCI ’17). Association for Computing Machinery, New York, NY, USA, Article 43, 12 pages. https://doi.org/10.1145/3098279.3098539

Digital Library

[15]

Fred D. Davis. 1989. Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Information Technology. MIS Quarterly 13, 3 (1989), 319–340. http://www.jstor.org/stable/249008

Digital Library

[16]

Alireza Fereidunian, Mohammad Ali Zamani, Bahar Fatah, Hamid Lesani, Caro Lucas, Payam Kharazmi, and Habib Torabi. 2010. Investigation of performance shaping factors in a practical human-automation interaction system. In 2010 IEEE International Conference on Systems, Man and Cybernetics. 1825–1831. https://doi.org/10.1109/ICSMC.2010.5642287

[17]

Claes Fornell and David F. Larcker. 1981. Evaluating Structural Equation Models with Unobservable Variables and Measurement Error. Journal of Marketing Research 18, 1 (1981), 39–50. http://www.jstor.org/stable/3151312

[18]

Dogan Gursoy, Oscar Hengxuan Chi, Lu Lu, and Robin Nunkoo. 2019. Consumers acceptance of artificially intelligent (AI) device use in service delivery. International Journal of Information Management 49 (2019), 157–169. https://doi.org/10.1016/j.ijinfomgt.2019.03.008

Digital Library

[19]

J.F. Hair, W.C. Black, R.E. Anderson, and B.J. Babin. 2018. Multivariate Data Analysis (8th ed.). Cengage.

[20]

Paige Hayes and Jason Wagner. 2018. Prepare for the voice revolution. https://www.pwc.com/us/en/services/consulting/library/consumer-intelligence-series/voice-assistants.html.

[21]

Li‐tze Hu and Peter M. Bentler. 1999. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal 6, 1 (1999), 1–55. https://doi.org/10.1080/10705519909540118

[22]

Dharun Lingam Kasilingam. 2020. Understanding the attitude and intention to use smartphone chatbots for shopping. Technology in Society 62 (2020), 101280. https://doi.org/10.1016/j.techsoc.2020.101280

[23]

Timothy Z Keith. 2019. Multiple regression and beyond: An introduction to multiple regression and structural equation modeling. Routledge. 62–63 pages.

[24]

Hee-Woong Kim, Hock Chuan Chan, and Sumeet Gupta. 2007. Value-based Adoption of Mobile Internet: An empirical investigation. Decision Support Systems 43, 1 (2007), 111–126. https://doi.org/10.1016/j.dss.2005.05.009 Mobile Commerce: Strategies, Technologies, and Applications.

Digital Library

[25]

Rex B Kline. 2015. Principles and practice of structural equation modeling. Guilford publications.

[26]

Sherrie Y. X. Komiak and Izak Benbasat. 2006. The Effects of Personalization and Familiarity on Trust and Adoption of Recommendation Agents. MIS Quarterly 30, 4 (2006), 941–960. http://www.jstor.org/stable/25148760

[27]

Sherrie Y. X. Komiak and Izak Benbasat. 2006. The effects of personalization and familiarity on trust and adoption of recommendation agents. MIS quarterly (2006), 941–960.

[28]

John D. Lee and Neville Moray. 1994. Trust, self-confidence, and operators’ adaptation to automation. International Journal of Human-Computer Studies 40, 1 (1994), 153–184. https://doi.org/10.1006/ijhc.1994.1007

Digital Library

[29]

Michael Leyer and Sabrina Schneider. 2019. ME, YOU OR AI? HOW DO WE FEEL ABOUT DELEGATION. In Proceedings of the 27th European Conference on Information Systems (ECIS). Stockholm & Uppsala, Sweden, 1825–1831. https://aisel.aisnet.org/ecis2019_rp/36

[30]

Leonardo Liberman and Dirk Boehe. 2011. Worldwide willingness to delegate and country labor quality. The International Journal of Human Resource Management 22, 17 (2011), 3477–3495. https://doi.org/10.1080/09585192.2011.610943

[31]

Brian Lubars and Chenhao Tan. 2019. Ask not what AI can do, but what AI should do: Towards a framework of task delegability. In Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and R. Garnett (Eds.). Vol. 32. Curran Associates, Inc.https://proceedings.neurips.cc/paper/2019/file/d67d8ab4f4c10bf22aa353e27879133c-Paper.pdf

[32]

Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA": The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (San Jose, California, USA) (CHI ’16). Association for Computing Machinery, New York, NY, USA, 5286–5297. https://doi.org/10.1145/2858036.2858288

Digital Library

[33]

Hasan Mahmud, A. K. M. Najmul Islam, Ranjan Kumar Mitra, and Ahmed Rizvan Hasan. 2022. The Impact of Functional and Psychological Barriers on Algorithm Aversion – An IRT Perspective. In The Role of Digital Technologies in Shaping the Post-Pandemic World, Savvas Papagiannidis, Eleftherios Alamanos, Suraksha Gupta, Yogesh K. Dwivedi, Matti Mäntymäki, and Ilias O. Pappas (Eds.). Springer International Publishing, Cham, 95–108.

[34]

Hasan Mahmud, A.K.M. Najmul Islam, Syed Ishtiaque Ahmed, and Kari Smolander. 2022. What influences algorithmic decision-making? A systematic literature review on algorithm aversion. Technological Forecasting and Social Change 175 (2022), 121390. https://doi.org/10.1016/j.techfore.2021.121390

[35]

Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An Integrative Model of Organizational Trust. The Academy of Management Review 20, 3 (1995), 709–734. http://www.jstor.org/stable/258792

[36]

Brian K. Miller and Marcia Simmering. 2020. Impact of Survey Design Features on Score Reliability. Collabra: Psychology 6, 1 (11 2020). https://doi.org/10.1525/collabra.17975 17975.

[37]

Jum C. Nunnally and Ira H. Bernstein. 1994. Psychometric Theory. McGraw-Hill Companies,Incorporated.

[38]

R. Parasuraman, T.B. Sheridan, and C.D. Wickens. 2000. A model for types and levels of human interaction with automation. IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans 30, 3 (2000), 286–297. https://doi.org/10.1109/3468.844354

Digital Library

[39]

Robert A. Peterson. 1994. A Meta-analysis of Cronbach’s Coefficient Alpha. Journal of Consumer Research 21, 2 (09 1994), 381–391.

[40]

Rajasshrie Pillai and Brijesh Sivathanu. 2020. Adoption of AI-based chatbots for hospitality and tourism. International Journal of Contemporary Hospitality Management (2020).

[41]

Yves Rosseel. 2012. lavaan: a brief user’s guide. https://users.ugent.be/ yrosseel/lavaan/lavaan2.pdf.

[42]

Albert Satorra and Peter M Bentler. 2001. A scaled difference chi-square test statistic for moment structure analysis. Psychometrika 66, 4 (2001), 507–514.

[43]

Justina Sidlauskiene. 2022. What Drives Consumers’ Decisions to Use Intelligent Agent Technologies? A Systematic Review. Journal of Internet Commerce 21, 4 (2022), 438–475. https://doi.org/10.1080/15332861.2021.1961192

[44]

Bergur Thormundsson. 2022. Virtual Assistant Technology – Statistics & Facts. https://www.statista.com/topics/5572/virtual-assistants/##topicOverview.

[45]

Milka Trajkova and Aqueasha Martin-Hammond. 2020. "Alexa is a Toy": Exploring Older Adults’ Reasons for Using, Limiting, and Abandoning Echo. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–13. https://doi.org/10.1145/3313831.3376760

Digital Library

[46]

Brahim Zarouali, Evert Van den Broeck, Michel Walrave, and Karolien Poels. 2018. Predicting Consumer Responses to a Chatbot on Facebook. Cyberpsychology, Behavior, and Social Networking 21, 8 (2018), 491–497. https://doi.org/10.1089/cyber.2017.0518 30036074.

Cited By

Bartsch ANeuberger CStark BKarnowski VMaurer MPentzold CQuandt TQuiring OSchemer C(2024)Epistemic authority in the digital public sphere. An integrative conceptual framework and research agendaCommunication Theory10.1093/ct/qtae02035:1(37-50)Online publication date: 5-Nov-2024
https://doi.org/10.1093/ct/qtae020

Index Terms

Expectation vs Reality in Users’ Willingness to Delegate to Digital Assistants
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI theory, concepts and models

Recommendations

Modeling users' acceptance of mobile services

The success of mobile services adoption hinges on their ability to cover user needs and attract consumer interest. The extant literature focuses on understanding the factors that might affect consumers' actual adoption of such services through their ...
Factors affecting users' decision to continue using mobile banking in Turkey

This study aims to identify factors affecting the decisions of users in Turkey to continue using mobile banking. A research model was developed to identify the relationships among the relevant constructs: continued usage intention, satisfaction, trust, ...
Predicting e-services adoption: a perceived risk facets perspective
Special issue on HCI and MIS

Internet-delivered e-services are increasingly being made available to consumers; however, little is known about how consumers evaluate them for potential adoption. Past Technology Adoption Research has focused primarily on the positive utility gains ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '23: Extended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems

April 2023

3914 pages

ISBN:9781450394222

DOI:10.1145/3544549

Editors:
Albrecht Schmidt
LMU Munich, Germany
,
Kaisa Väänänen
Tampere University, Finland
,
Tesh Goyal
Google Research, USA
,
Per Ola Kristensson
University of Cambridge, UK
,
Anicia Peters
University of Namibia, Namibia

Copyright © 2023 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 19 April 2023

Check for updates

Author Tags

Qualifiers

Work in progress
Research
Refereed limited

Conference

CHI '23

Sponsor:

SIGCHI

CHI '23: CHI Conference on Human Factors in Computing Systems

April 23 - 28, 2023

Hamburg, Germany

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
440
Total Downloads

Downloads (Last 12 months)246
Downloads (Last 6 weeks)42

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bartsch ANeuberger CStark BKarnowski VMaurer MPentzold CQuandt TQuiring OSchemer C(2024)Epistemic authority in the digital public sphere. An integrative conceptual framework and research agendaCommunication Theory10.1093/ct/qtae02035:1(37-50)Online publication date: 5-Nov-2024
https://doi.org/10.1093/ct/qtae020

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Table of Conten