short-paper

Trust and Reliance Based on System Accuracy

Authors:

Shlomo Berkovsky,

Fang ChenAuthors Info & Claims

UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization

Pages 223 - 227

https://doi.org/10.1145/2930238.2930290

Published: 13 July 2016 Publication History

Abstract

Trust plays an important role in various user-facing systems and applications. It is particularly important in the context of decision support systems, where the system's output serves as one of the inputs for the users' decision making processes. In this work, we study the dynamics of explicit and implicit user trust in a simulated automated quality monitoring system, as a function of the system accuracy. We establish that users correctly perceive the accuracy of the system and adjust their trust accordingly.

References

[1]

J. D. Lee and N. Moray, "Trust, self-confidence, and operators' adaptation to automation," Int. J. Hum.-Comput. Stud., vol. 40, no. 1, pp. 153--184, Jan. 1994.

Digital Library

[2]

K. A. Hoff and M. Bashir, "Trust in Automation Integrating Empirical Evidence on Factors That Influence Trust," Hum. Factors J. Hum. Factors Ergon. Soc., vol. 57, no. 3, pp. 407--434, May 2015.

[3]

B. M. Muir, "Trust in automation: Part I. Theoretical issues in the study of trust and human intervention in automated systems," Ergonomics, vol. 37, no. 11, pp. 1905--1922, Nov. 1994.

[4]

J. M. McGuirl and N. B. Sarter, "Supporting Trust Calibration and the Effective Use of Decision Aids by Presenting Dynamic System Confidence Information," Hum. Factors J. Hum. Factors Ergon. Soc., vol. 48, no. 4, pp. 656--665, Dec. 2006.

[5]

W. Wang and I. Benbasat, "Attributions of Trust in Decision Support Technologies: A Study of Recommendation Agents for E-Commerce," J. Manag. Inf. Syst., vol. 24, no. 4, pp. 249--273, Apr. 2008.

Digital Library

[6]

J. Zhou, "Transparent Machine Learning-Revealing Internal States of Machine Learning," in Proceedings of IUI2013 Workshop on Interactive Machine Learning, Santa Monica, CA, 2013.

[7]

J. D. Lee and K. A. See, "Trust in Automation: Designing for Appropriate Reliance," Hum. Factors J. Hum. Factors Ergon. Soc., vol. 46, no. 1, pp. 50--80, Mar. 2004.

[8]

D. M. Rousseau, S. B. Sitkin, R. S. Burt, and C. Camerer, "Not So Different After All: A Cross-Discipline View Of Trust," Acad. Manage. Rev., vol. 23, no. 3, pp. 393--404, Jul. 1998.

[9]

R. C. Mayer, J. H. Davis, and F. D. Schoorman, "An Integrative Model Of Organizational Trust," Acad. Manage. Rev., vol. 20, no. 3, pp. 709--734, Jul. 1995.

[10]

J.-Y. Jian, A. M. Bisantz, and C. G. Drury, "Foundations for an Empirically Determined Scale of Trust in Automated Systems," Int. J. Cogn. Ergon., vol. 4, no. 1, pp. 53--71, Mar. 2000.

[11]

J. Rotter, "An new scale for the measurement of interpersonal trust," J. Pers., vol. 35, no. 4, pp. 651--665, 1967.

[12]

C. L. Scott, "Interpersonal Trust: A Comparison of Attitudinal and Situational Factors," Hum. Relat., vol. 33, no. 11, pp. 805--812, Nov. 1980.

[13]

I. L. Singh, R. Molloy, and R. Parasuraman, "Automation-Induced Complacency: Development of the Complacency-Potential Rating Scale," Int. J. Aviat. Psychol., vol. 3, no. 2, pp. 111--122, Apr. 1993.

[14]

P. Madhavan and D. A. Wiegmann, "Similarities and differences between human-human and human-automation trust: an integrative review," Theor. Issues Ergon. Sci., vol. 8, no. 4, pp. 277--301, Jul. 2007.

[15]

P. C. Earley, "Computer-generated performance feedback in the magazine-subscription industry," Organ. Behav. Hum. Decis. Process., vol. 41, no. 1, pp. 50--64, Feb. 1988.

[16]

B. J. Dietvorst, J. P. Simmons, and C. Massey, "Algorithm aversion: People erroneously avoid algorithms after seeing them err," J. Exp. Psychol. Gen., vol. 144, no. 1, pp. 114--126, 2015.

[17]

M. T. Dzindolet, L. G. Pierce, H. P. Beck, and L. A. Dawe, "The Perceived Utility of Human and Automated Aids in a Visual Detection Task," Hum. Factors J. Hum. Factors Ergon. Soc., vol. 44, no. 1, pp. 79--94, Mar. 2002.

[18]

S. Berkovsky, J. Freyne, and H. Oinas-Kukkonen, Eds., "Influencing Individually: Fusing Personalization and Persuasion," ACM Trans Interact Intell Syst, vol. 2, no. 2, pp. 9:1--9:8, Jun. 2012.

Digital Library

[19]

S. Merritt and D. Ilgen, "Not all trust is created equal: Dispositional and history-based trust in human-automation interactions," Hum. Factors, vol. 50, no. 2, pp. 194--210, 2008.

Cited By

Martens MDe Wolf RDe Marez L(2024)Trust in algorithmic decision-making systems in health: A comparison between ADA health and IBM Watson.Cyberpsychology: Journal of Psychosocial Research on Cyberspace10.5817/CP2024-1-518:1Online publication date: 1-Feb-2024
https://doi.org/10.5817/CP2024-1-5
Hinrichs MBui TSchneegass S(2024)Exploring the Effects of User Input and Decision Criteria Control on Trust in a Decision Support Tool for Spare Parts Inventory ManagementProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701585(313-323)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701585
Pareek SVelloso EGoncalves J(2024)Trust Development and Repair in AI-Assisted Decision-Making during Complementary ExpertiseProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658924(546-561)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658924
Show More Cited By

Index Terms

Trust and Reliance Based on System Accuracy

Recommendations

User Trust Dynamics: An Investigation Driven by Differences in System Performance
IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

Trust is a key factor affecting the way people rely on automated systems. On the other hand, system performance has comprehensive implications on a user's trust variations. This paper examines systems of varied levels of accuracy, in order to reveal the ...
Effect of Wait Time on Trust and Reliance in Human-Robot Interaction
HRI '24: Companion of the 2024 ACM/IEEE International Conference on Human-Robot Interaction

The design of trust is a critical issue in human-robot interaction (HRI). This study investigated how waiting time for a robot's action affects trust and reliance on a robot. In Experiment 1, an online study manipulated the wait time for the task partner'...
Finding underlying factors for reciprocal trust formation
RACS '15: Proceedings of the 2015 Conference on research in adaptive and convergent systems

Trust in social media is an expression of faith that a user (called trustor) has a mind to accept the information provided by someone else (called trustee). While only a trustor in a one-way trust relationship tends to receive the information of his ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP '16: Proceedings of the 2016 Conference on User Modeling Adaptation and Personalization

July 2016

366 pages

ISBN:9781450343688

DOI:10.1145/2930238

General Chairs:
Julita Vassileva
University of Saskatchewan, Canada
,
James Blustein
Dalhousie University, Canada
,
Program Chairs:
Lora Aroyo
VU Amsterdam, the Netherlands
,
Sidney D'Mello
University of Notre Dame, USA

Copyright © 2016 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 July 2016

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

NICTA
Asian Office of Aerospace Research & Development (AOARD)

Conference

UMAP '16

Sponsor:

UMAP '16: User Modeling, Adaptation and Personalization Conference

July 13 - 17, 2016

Nova Scotia, Halifax, Canada

Acceptance Rates

UMAP '16 Paper Acceptance Rate 21 of 123 submissions, 17%;

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

19
Total Citations
View Citations
428
Total Downloads

Downloads (Last 12 months)44
Downloads (Last 6 weeks)4

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Martens MDe Wolf RDe Marez L(2024)Trust in algorithmic decision-making systems in health: A comparison between ADA health and IBM Watson.Cyberpsychology: Journal of Psychosocial Research on Cyberspace10.5817/CP2024-1-518:1Online publication date: 1-Feb-2024
https://doi.org/10.5817/CP2024-1-5
Hinrichs MBui TSchneegass S(2024)Exploring the Effects of User Input and Decision Criteria Control on Trust in a Decision Support Tool for Spare Parts Inventory ManagementProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701585(313-323)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1145/3701571.3701585
Pareek SVelloso EGoncalves J(2024)Trust Development and Repair in AI-Assisted Decision-Making during Complementary ExpertiseProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658924(546-561)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658924
Guo ZWu YHartline JHullman J(2024)A Decision Theoretic Framework for Measuring AI RelianceProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency10.1145/3630106.3658901(221-236)Online publication date: 3-Jun-2024
https://dl.acm.org/doi/10.1145/3630106.3658901
Elhamdadi HStefkovics ABeyer JMoerth EPfister HBearfield CNobre C(2024)Vistrust: a Multidimensional Framework and Empirical Study of Trust in Data VisualizationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332657930:1(348-358)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TVCG.2023.3326579
Kim AYang MZhang J(2023) When Algorithms Err: Differential Impact of Early vs. Late Errors on Users’ Reliance on AlgorithmsACM Transactions on Computer-Human Interaction10.1145/355788930:1(1-36)Online publication date: 18-Mar-2023
https://dl.acm.org/doi/10.1145/3557889
Park JLee KKim H(2023)Integrated Recognition Assistant Framework Based on Deep Learning for Autonomous Driving: Human-Like Restoring Damaged Road Sign InformationInternational Journal of Human–Computer Interaction10.1080/10447318.2023.220427440:15(3982-4002)Online publication date: 27-Apr-2023
https://doi.org/10.1080/10447318.2023.2204274
Brachman MAshktorab ZDesmond MDuesterwald EDugan CJoshi NPan QSharma A(2022)Reliance and Automation for Human-AI Collaborative Data Labeling Conflict ResolutionProceedings of the ACM on Human-Computer Interaction10.1145/35552126:CSCW2(1-27)Online publication date: 11-Nov-2022
https://dl.acm.org/doi/10.1145/3555212
Panigutti CBeretta AGiannotti FPedreschi D(2022)Understanding the impact of explanations on advice-taking: a user study for AI-based clinical Decision Support SystemsProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502104(1-9)Online publication date: 29-Apr-2022
https://dl.acm.org/doi/10.1145/3491102.3502104
Kaptchuk GGoldstein DHargittai EHofman JRedmiles E(2022)How Good is Good Enough? Quantifying the Impact of Benefits, Accuracy, and Privacy on Willingness to Adopt COVID-19 Decision AidsDigital Threats: Research and Practice10.1145/34883073:3(1-18)Online publication date: 26-Mar-2022
https://dl.acm.org/doi/10.1145/3488307
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten