skip to main content
10.1145/1234772.1234775acmconferencesArticle/Chapter ViewAbstractPublication PageschimitConference Proceedingsconference-collections
Article

Deciding when to trust automation in a policy-based city management game: policity

Published: 30 March 2007 Publication History

Abstract

As businesses and governments strive to improve productivity and deploy more elaborate IT systems, the need for complex systems management grows. Completely automated systems are not yet a reality, so the benefits that automation offers can only be achieved through collaboration with human operators. The question, however, is what factors influence decisions about how this human-computer relationship will be coordinated? A great deal of research has pointed to trust and perceived reliability as key factors in whether automation will be properly used, misused, or disused in systems management. To explore this question, we conducted an experiment in which an automated decision aid presented suggestions or policies to participants while they managed a simulated city (i.e., Policity). The goal was to maximize the health of the city's population by adding hospitals, housing, businesses and other facilities and services. Participants were randomly assigned to conditions where the decision aid performed with varying (i.e., high or low) reliability levels. Results showed that users' perception of the decision aid's reliability directly influenced their trust in the decision aid. Consequently, the relationship between users' perceived reliability and the decision aid's (actual) reliability had a direct effect on human performance. Population health suffered when the decision aid's suggestions were disused and misused compared to when they were appropriately used. Additional results and implications are discussed. ACM Classification

References

[1]
Campbell, C. S., Kandogan, E., November, A., Barrett, R., Maglio, P. P. (2005). Policity: An Experimental Evaluation of Policy-Based Administration in a City Simulation. IEEE 6th International Workshop on Policies for Distributed Systems and Networks (Policy 2005).
[2]
Dzindolet, M. T., Beck, H. P. & Pierce, L. G. Encouraging human operators to appropriately rely in automated decision aids. Proceedings of the 2000 Command and Control Research and Technology Symposium, Monterey, CA: Department of Defense Cooperative Research Program (2000).
[3]
Helmreich, R. L. Cockpit management attitudes. Human Factors, 5, (1984), 583--589.
[4]
Kandogan, E., Campbell, C. S., Khooshabeh, P., Bailey, J., Maglio, P. P. Policy-based Management of an E-commerce Business Simulation: An Experimental Study. 3rd IEEE International Conference on Autonomic Computing (ICAC 06), 13--16 June 2006, 33--42.
[5]
Littlewood B., Strigini L. A discussion of practices for enhancing diversity in software designs, DISPO project technical report, Centre for Software Reliability, City University, 2000.
[6]
Madsen, M. and Gregor, S. Measuring human-computer trust. In G. Gable and M. Viatle (Eds.). Proceedings of the 11th Australasian Conference on Information Systems, Brisbane, 6--8 Dec., (2000), p. 53.
[7]
McClumpha, A. & James, M. Understanding automated aircraft. In M. Mouloua & R. Parasuraman (Eds.) Human performance in automated systems: Current research and trends (pp. 183--190), Hillsdale, NJ:LEA, (1994).
[8]
Muir, B. M., & Moray, N. Trust in automation: 2. Experimental studies of trust and human intervention in automated systems. Ergonomics, 37, (1996), 1905--1922.
[9]
Oliner, S. D. & Sichel, D. E. Information technology and productivity: Where are we now and where are we going? Economic Review 87(3), (2002). 15--44.
[10]
Parasuraman, R. & Riley, V. Humans and automation: Use, misuse, disuse, and abuse. Human Factors, 39, (1997), 230--253.
[11]
Reeves, W. Learner-centered design. California: Sage Publications, Inc (1999).
[12]
Rempel, J. K., Holmes, J. G., & Zanna, M. P. Trust in close relationships. Journal of Personality and Social Psychology, 49(1), (1985), 95--112.
[13]
Sheridan, T. B. Trustworthiness of command and control systems. Conference on Analysis; Design and Evaluation of Man-Machine Systems, (1988), 427--431.

Cited By

View all
  • (2024)Exploring the Effects of User Input and Decision Criteria Control on Trust in a Decision Support Tool for Spare Parts Inventory ManagementProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701585(313-323)Online publication date: 1-Dec-2024
  • (2023) Trustworthy artificial intelligence and the European Union AI act: On the conflation of trustworthiness and acceptability of risk Regulation & Governance10.1111/rego.1251218:1(3-32)Online publication date: 6-Feb-2023
  • (2021)Designing Trustworthy User InterfacesProceedings of the 33rd Australian Conference on Human-Computer Interaction10.1145/3520495.3520525(182-189)Online publication date: 30-Nov-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHIMIT '07: Proceedings of the 2007 symposium on Computer human interaction for the management of information technology
March 2007
124 pages
ISBN:9781595936356
DOI:10.1145/1234772
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 March 2007

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. IT service delivery
  2. algorithms
  3. automation
  4. decision aids
  5. human factors
  6. information technology management
  7. policy-based interaction
  8. productivity
  9. reliability
  10. services research
  11. trust

Qualifiers

  • Article

Conference

CHiMiT07
Sponsor:

Acceptance Rates

CHIMIT '07 Paper Acceptance Rate 11 of 34 submissions, 32%;
Overall Acceptance Rate 15 of 43 submissions, 35%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)10
  • Downloads (Last 6 weeks)1
Reflects downloads up to 14 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Exploring the Effects of User Input and Decision Criteria Control on Trust in a Decision Support Tool for Spare Parts Inventory ManagementProceedings of the International Conference on Mobile and Ubiquitous Multimedia10.1145/3701571.3701585(313-323)Online publication date: 1-Dec-2024
  • (2023) Trustworthy artificial intelligence and the European Union AI act: On the conflation of trustworthiness and acceptability of risk Regulation & Governance10.1111/rego.1251218:1(3-32)Online publication date: 6-Feb-2023
  • (2021)Designing Trustworthy User InterfacesProceedings of the 33rd Australian Conference on Human-Computer Interaction10.1145/3520495.3520525(182-189)Online publication date: 30-Nov-2021
  • (2021)How to Evaluate Trust in AI-Assisted Decision Making? A Survey of Empirical MethodologiesProceedings of the ACM on Human-Computer Interaction10.1145/34760685:CSCW2(1-39)Online publication date: 18-Oct-2021
  • (2019)‘If You Agree with Me, Do I Trust You?’: An Examination of Human-Agent Trust from a Psychological PerspectiveIntelligent Systems and Applications10.1007/978-3-030-29513-4_73(994-1013)Online publication date: 24-Aug-2019
  • (2018)Inaccuracy versus volatility – Which is the lesser evil in battery electric vehicles?Transportation Research Part F: Traffic Psychology and Behaviour10.1016/j.trf.2018.07.01658(855-870)Online publication date: Oct-2018
  • (2017)Personal Influences on Dynamic Trust Formation in Human-Agent InteractionProceedings of the 5th International Conference on Human Agent Interaction10.1145/3125739.3125749(233-243)Online publication date: 17-Oct-2017
  • (2014)Trust in AutomationHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/001872081454757057:3(407-434)Online publication date: 2-Sep-2014
  • (2011)On the roles of policies in computer systems managementInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2011.01.00469:6(351-361)Online publication date: 1-Jun-2011

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media