research-article

Increasing User Trust in Optimisation through Feedback and Interaction

Authors:

Guido TackAuthors Info & Claims

ACM Transactions on Computer-Human Interaction, Volume 29, Issue 5

Article No.: 42, Pages 1 - 34

https://doi.org/10.1145/3503461

Published: 06 January 2023 Publication History

Abstract

User trust plays a key role in determining whether autonomous computer applications are relied upon. It will play a key role in the acceptance of emerging AI applications such as optimisation. Two important factors known to affect trust are system transparency, i.e., how well the user understands how the system works, and system performance. However, in the case of optimisation, it is difficult for the end-user to understand the underlying algorithms or to judge the quality of the solution. Through two controlled user studies, we explore whether the user is better able to calibrate their trust in the system when: (a) They are provided feedback on the system operation in the form of visualisation of intermediate solutions and their quality; (b) They can interactively explore the solution space by modifying the solution returned by the system. We found that showing intermediate solutions can lead to over-trust, while interactive exploration leads to more accurately calibrated trust.

References

[1]

Ashraf Abdul, Jo Vermeulen, Danding Wang, Brian Y. Lim, and Mohan Kankanhalli. 2018. Trends and trajectories for explainable, accountable and intelligible systems: An hci research agenda. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–18.

Digital Library

[2]

David Anderson, Emily Anderson, Neal Lesh, Joe Marks, Brian Mirtich, David Ratajczak, and Kathy Ryall. 2000. Human-guided simple search. In Proceedings of the AAAI/IAAI. 209–216.

[3]

Alejandro Barredo Arrieta, Natalia Díaz-Rodríguez, Javier Del Ser, Adrien Bennetot, Siham Tabik, Alberto Barbado, Salvador García, Sergio Gil-López, Daniel Molina, Richard Benjamins, et al. 2020. Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion 58 (2020), 82–115.

Digital Library

[4]

Donovan Artz and Yolanda Gil. 2007. A survey of trust in computer science and the semantic web. Web Semantics: Science, Services and Agents on the World Wide Web 5, 2 (2007), 58–71.

Digital Library

[5]

Nathan R. Bailey and Mark W. Scerbo. 2007. Automation-induced complacency for monitoring highly reliable systems: The role of task complexity, system experience, and operator trust. Theoretical Issues in Ergonomics Science 8, 4 (2007), 321–348.

[6]

Gilles Bailly, Antti Oulasvirta, Timo Kötzing, and Sabrina Hoppe. 2013. Menuoptimizer: Interactive optimization of menu systems. In Proceedings of the 26th Annual ACM Symposium on User Interface Software and Technology. ACM, 331–342.

Digital Library

[7]

Matthew Brehmer, Bongshin Lee, Benjamin Bach, Nathalie Henry Riche, and Tamara Munzner. 2017. Timelines revisited: A design space and considerations for expressive storytelling. IEEE Transactions on Visualization and Computer Graphics 23, 9 (2017), 2151–2164.

Digital Library

[8]

Pamela Briggs, B. Burford, and C. Dracup. 1998. Modelling self-confidence in users of a computer-based system showing unrepresentative design. International Journal of Human-Computer Studies 49, 5 (1998), 717–742.

Digital Library

[9]

Eric Brochu, Tyson Brochu, and Nando de Freitas. 2010. A Bayesian interactive optimization approach to procedural animation design. In Proceedings of the 2010 ACM SIGGRAPH/Eurographics Symposium on Computer Animation. Eurographics Association, 103–112.

Digital Library

[10]

Keith A. Butler, Jiajie Zhang, Chris Esposito, Ali Bahrami, Ron Hebron, and David Kieras. 2007. Work-centered design: A case study of a mixed-initiative scheduler. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 747–756.

Digital Library

[11]

Rich Caruana, Yin Lou, Johannes Gehrke, Paul Koch, Marc Sturm, and Noemie Elhadad. 2015. Intelligible models for healthcare: Predicting pneumonia risk and hospital 30-day readmission. In Proceedings of the 21th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1721–1730.

Digital Library

[12]

Timo Christophersen and Udo Konradt. 2011. Reliability, validity, and sensitivity of a single-item measure of online store usability. International Journal of Human-Computer Studies 69, 4 (2011), 269–280.

Digital Library

[13]

Marvin S. Cohen, Raja Parasuraman, and Jared T. Freeman. 1998. Trust in decision aids: A model and its training implications. In Proceedings of the Command and Control Research and Technology Symposium. Citeseer.

[14]

Cynthia L. Corritore, Beverly Kracher, and Susan Wiedenbeck. 2003. On-line trust: Concepts, evolving themes, a model. International Journal of Human-computer Studies 58, 6 (2003), 737–758.

Digital Library

[15]

Dan Cosley, Shyong K. Lam, Istvan Albert, Joseph A. Konstan, and John Riedl. 2003. Is seeing believing? How recommender system interfaces affect users’ opinions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 585–592.

Digital Library

[16]

Teodor G. Crainic and Gilbert Laporte. 2012. Fleet Management and Logistics. Springer Science & Business Media.

[17]

Mary L. Cummings, Jessica J. Marquez, and Nicholas Roy. 2012. Human-automated path planning optimization and decision support. International Journal of Human-Computer Studies 70, 2 (2012), 116–128.

Digital Library

[18]

Kristijonas Čyras, Dimitrios Letsios, Ruth Misener, and Francesca Toni. 2019. Argumentation for explainable scheduling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 2752–2759.

[19]

Ewart de Visser and Raja Parasuraman. 2011. Adaptive aiding of human-robot teaming: Effects of imperfect automation on performance, trust, and workload. Journal of Cognitive Engineering and Decision Making 5, 2 (2011), 209–231.

[20]

Mary T. Dzindolet, Scott A. Peterson, Regina A. Pomranky, Linda G. Pierce, and Hall P. Beck. 2003. The role of trust in automation reliance. International Journal of Human-computer Studies 58, 6 (2003), 697–718.

Digital Library

[21]

Nasser A. El-Sherbeny. 2010. Vehicle routing with time windows: An overview of exact, heuristic and metaheuristic methods. Journal of King Saud University-Science 22, 3 (2010), 123–131.

[22]

Andy Field, Jeremy Miles, and Zoë Field. 2012. Discovering Statistics Using R. Sage publications.

Digital Library

[23]

B. J. Fogg and Hsiang Tseng. 1999. The elements of computer credibility. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 80–87.

Digital Library

[24]

Gecode Team. 2016. Gecode: A Generic Constraint Development Environment. Retrieved from http://www.gecode.org.

[25]

David Gefen, Elena Karahanna, and Detmar W. Straub. 2003. Trust and TAM in online shopping: An integrated model. MIS Quarterly 27, 1 (2003), 51–90.

[26]

Alyssa Glass, Deborah L. McGuinness, and Michael Wolverton. 2008. Toward establishing trust in adaptive agents. In Proceedings of the 13th International Conference on Intelligent User Interfaces. ACM, 227–236.

Digital Library

[27]

Jason L. Harman, John O’Donovan, Tarek Abdelzaher, and Cleotilde Gonzalez. 2014. Dynamics of human trust in recommender systems. In Proceedings of the 8th ACM Conference on Recommender Systems. ACM, 305–308.

Digital Library

[28]

Monika Hengstler, Ellen Enkel, and Selina Duelli. 2016. Applied artificial intelligence and trust? The case of autonomous vehicles and medical assistance devices. Technological Forecasting and Social Change 105 (2016), 105–120.

[29]

Jonathan L. Herlocker, Joseph A. Konstan, and John Riedl. 2000. Explaining collaborative filtering recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work. 241–250.

Digital Library

[30]

Kevin Anthony Hoff and Masooda Bashir. 2015. Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors 57, 3 (2015), 407–434.

[31]

Robert R. Hoffman, John D. Lee, David D. Woods, Nigel Shadbolt, Janet Miller, and Jeffrey M. Bradshaw. 2009. The dynamics of trust in cyberdomains. IEEE Intelligent Systems6 (2009), 5–11.

Digital Library

[32]

Jiun-Yin Jian, Ann M. Bisantz, and Colin G. Drury. 2000. Foundations for an empirically determined scale of trust in automated systems. International Journal of Cognitive Ergonomics 4, 1 (2000), 53–71.

[33]

Ashish Kapoor, Bongshin Lee, Desney Tan, and Eric Horvitz. 2010. Interactive optimization for steering machine classification. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1343–1352.

Digital Library

[34]

Been Kim. 2015. Interactive and Interpretable Machine Learning Models for Human Machine Collaboration. Ph.D. Dissertation. Massachusetts Institute of Technology.

[35]

Jinwoo Kim and Jae Yun Moon. 1998. Designing towards emotional usability in customer interfaces? Trustworthiness of cyber-banking system interfaces. Interacting with Computers 10, 1 (1998), 1–29.

[36]

Suresh Nanda Kumar and Ramasamy Panneerselvam. 2012. A survey on the vehicle routing problem and its variants. Intelligent Information Management 4, 3 (2012), 66.

[37]

John Lee and Neville Moray. 1992. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35, 10 (1992), 1243–1270.

[38]

John D. Lee and Katrina A. See. 2004. Trust in automation: Designing for appropriate reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society 46, 1 (2004), 50–80.

[39]

Matthew K. O. Lee and Efraim Turban. 2001. A trust model for consumer internet shopping. International Journal of Electronic Commerce 6, 1 (2001), 75–91.

Digital Library

[40]

Benjamin Letham, Cynthia Rudin, Tyler H. McCormick, David Madigan, et al. 2015. Interpretable classifiers using rules and bayesian analysis: Building a better stroke prediction model. The Annals of Applied Statistics 9, 3 (2015), 1350–1371.

[41]

Yung-Ming Li and Yung-Shao Yeh. 2010. Increasing trust in mobile commerce through design aesthetics. Computers in Human Behavior 26, 4 (2010), 673–684.

Digital Library

[42]

Zachary C. Lipton. 2018. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue 16, 3 (2018), 31–57.

Digital Library

[43]

Jie Liu, Tim Dwyer, Kim Marriott, Jeremy Millar, and Annette Haworth. 2018. Understanding the relationship between interactive optimisation and visual analytics in the context of prostate brachytherapy. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 319–329.

[44]

Jie Liu, Tim Dwyer, Guido Tack, Samuel Gratzl, and Kim Marriott. 2020. Supporting the problem-solving loop: Designing highly interactive optimisation systems. IEEE Transactions on Visualization and Computer Graphics 27, 2 (2020), 1764–1774.

[45]

D. Harrison McKnight and Norman L. Chervany. 2000. What is trust? A conceptual analysis and an interdisciplinary model. AMCIS 2000 Proceedings (2000), 382.

[46]

David Meignan, Jean-Marc Frayret, and Gilles Pesant. 2011. An interactive heuristic approach for the P-forest problem. In Proceedings of the 2011 IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 1009–1013.

[47]

David Meignan, Sigrid Knust, Jean-Marc Frayret, Gilles Pesant, and Nicolas Gaud. 2015. A review and taxonomy of interactive optimization methods in operations research. ACM Transactions on Interactive Intelligent Systems (TiiS) 5, 3 (2015), 17.

[48]

Neville Moray, Toshiyuki Inagaki, and Makoto Itoh. 2000. Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied 6, 1 (2000), 44.

[49]

Bonnie M. Muir. 1987. Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies 27, 5–6 (1987), 527–539.

Digital Library

[50]

Bonnie M. Muir and Neville Moray. 1996. Trust in automation. Part II. Experimental studies of trust and human intervention in a process control simulation. Ergonomics 39, 3 (1996), 429–460.

[51]

Nicholas Nethercote, Peter J. Stuckey, Ralph Becket, Sebastian Brand, Gregory J. Duck, and Guido Tack. 2007. MiniZinc: Towards a standard CP modelling language. In Proceedings of the Principles and Practice of Constraint Programming – CP 2007, Christian Bessière (Ed.). Springer, Berlin, 529–543.

[52]

Barry O’Callaghan, Barry O’Sullivan, and Eugene C. Freuder. 2005. Generating corrective explanations for interactive constraint satisfaction. In Proceedings of the International Conference on Principles and Practice of Constraint Programming. Springer, 445–459.

Digital Library

[53]

Raja Parasuraman and Victor Riley. 1997. Humans and automation: Use, misuse, disuse, abuse. Human Factors 39, 2 (1997), 230–253.

[54]

James Patten and Hiroshi Ishii. 2007. Mechanical constraints as computational constraints in tabletop tangible interfaces. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 809–818.

Digital Library

[55]

Wolter Pieters. 2011. Explanation and trust: What to tell the user in security and AI? Ethics and Information Technology 13, 1 (2011), 53–64.

Digital Library

[56]

Zening Qu and Jessica Hullman. 2018. Keeping multiple views consistent: Constraints, validations, and exceptions in visualization authoring. IEEE Transactions on Visualization and Computer Graphics 24, 1 (2018), 468–477.

[57]

Eric D. Ragan, Alex Endert, Jibonananda Sanyal, and Jian Chen. 2016. Characterizing provenance in visualization and data analysis: An organizational framework of provenance types and purposes. IEEE Transactions on Visualization and Computer Graphics 22, 1 (2016), 31–40.

Digital Library

[58]

Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. “Why should I trust you?”: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. ACM, 1135–1144.

Digital Library

[59]

Quentin Roy, Futian Zhang, and Daniel Vogel. 2019. Automation accuracy is good, but high controllability may be better. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–8.

Digital Library

[60]

Kristin E. Schaefer, Deborah R. Billings, James L. Szalma, Jeffrey K. Adams, Tracy L. Sanders, Jessie Y. Chen, and Peter A. Hancock. 2014. A Meta-analysis of Factors Influencing the Development of Trust in Automation: Implications for Human-robot Interaction. Technical Report. Army Research Lab Aberdeen Proving Ground Md Human Research and Engineering Directorate.

[61]

Kristin E. Schaefer, Jessie Y. C. Chen, James L. Szalma, and Peter A. Hancock. 2016. A meta-analysis of factors influencing the development of trust in automation: Implications for understanding autonomy in future systems. Human Factors 58, 3 (2016), 377–400.

[62]

Stacey D. Scott, Neal Lesh, and Gunnar W. Klau. 2002. Investigating human-computer optimization. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 155–162.

Digital Library

[63]

Mitsuhiro Shibuya, Hajime Kita, and Shigenobu Kobayashi. 1999. Integration of multi-objective and interactive genetic algorithms and its application to animation design. In Proceedings of the IEEE International Conference on Systems, Man, and Cybernetics. IEEE, 646–651.

[64]

Keng Siau and Weiyu Wang. 2018. Building trust in artificial intelligence, machine learning, and robotics. Cutter Business Technology Journal 31, 2 (2018), 47–53.

[65]

Christian Thieke, Karl-Heinz Küfer, Michael Monz, Alexander Scherrer, Fernando Alonso, Uwe Oelfke, Peter E. Huber, Jürgen Debus, and Thomas Bortfeld. 2007. A new concept for interactive radiotherapy planning with multicriteria optimization: First clinical evaluation. Radiotherapy and Oncology 85, 2 (2007), 292–298.

[66]

Paolo Toth and Daniele Vigo (Eds.). 2001. The Vehicle Routing Problem. Society for Industrial and Applied Mathematics, Philadelphia, PA.

Digital Library

[67]

Paolo Toth and Daniele Vigo. 2002. The Vehicle Routing Problem. SIAM.

[68]

Frank MF Verberne, Jaap Ham, and Cees J. H. Midden. 2012. Trust in smart systems: Sharing driving goals and giving information to increase trustworthiness and acceptability of smart systems in cars. Human Factors 54, 5 (2012), 799–810.

Cited By

Damaševičius R(2024)Comment on Novozhilova et al. More Capable, Less Benevolent: Trust Perceptions of AI Systems across Societal Contexts. Mach. Learn. Knowl. Extr. 2024, 6, 342–366Machine Learning and Knowledge Extraction10.3390/make60300816:3(1667-1669)Online publication date: 22-Jul-2024
https://doi.org/10.3390/make6030081
Bahari GMutambik IAlmuqrin AAlharbi Z(2024)Trust: How It Affects the Use of Telemedicine in Improving Access to Assistive Technology to Enhance Healthcare ServicesRisk Management and Healthcare Policy10.2147/RMHP.S469324Volume 17(1859-1873)Online publication date: Jul-2024
https://doi.org/10.2147/RMHP.S469324
Lim CPrendez DBoyle LRajivan P(2024)The Impact of Cybersecurity Attacks on Human Trust in Autonomous Vehicle OperationsHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/00187208241283321Online publication date: 18-Sep-2024
https://doi.org/10.1177/00187208241283321
Show More Cited By

Index Terms

Increasing User Trust in Optimisation through Feedback and Interaction
1. Applied computing
  1. Operations research
    1. Decision analysis
      1. Multi-criterion optimization and decision-making
2. Human-centered computing
  1. Human computer interaction (HCI)
  2. Visualization

Recommendations

Examining Mobile Banking User Trust: A Tripartite Perspective

Building users' trust is crucial to alleviating their perceived risk and facilitating their usage of mobile banking. Drawing on a tripartite perspective of transference-based, personality-based and self-perception-based determinants, this research ...
Risk, trust, and the interaction of perceived ease of use and behavioral control in predicting consumers use of social media for transactions

There has been continued debate regarding competing models with respect to predicting use of social networking services. In this research the authors conceptualize and empirically test a model that combines constructs from the Technology Acceptance ...
National culture and consumer trust in e-commerce

The article examines how culture influences trust in e-commerce.Disposition to trust is a significant predictor of perceived trustworthiness.Disposition to trust mediates effects between national culture and trustworthiness.Long-term orientation and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Computer-Human Interaction

ACM Transactions on Computer-Human Interaction Volume 29, Issue 5

October 2022

453 pages

ISSN:1073-0516

EISSN:1557-7325

DOI:10.1145/3561950

Editor:
Kristina Höök
KTH Royal Institute of Technology, Sweden

Issue’s Table of Contents

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 06 January 2023

Online AM: 22 July 2022

Accepted: 30 November 2021

Revised: 29 September 2021

Received: 28 June 2019

Published in TOCHI Volume 29, Issue 5

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Refereed

Funding Sources

ICT Centre for Excellence Program

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
1,088
Total Downloads

Downloads (Last 12 months)419
Downloads (Last 6 weeks)51

Reflects downloads up to 03 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Damaševičius R(2024)Comment on Novozhilova et al. More Capable, Less Benevolent: Trust Perceptions of AI Systems across Societal Contexts. Mach. Learn. Knowl. Extr. 2024, 6, 342–366Machine Learning and Knowledge Extraction10.3390/make60300816:3(1667-1669)Online publication date: 22-Jul-2024
https://doi.org/10.3390/make6030081
Bahari GMutambik IAlmuqrin AAlharbi Z(2024)Trust: How It Affects the Use of Telemedicine in Improving Access to Assistive Technology to Enhance Healthcare ServicesRisk Management and Healthcare Policy10.2147/RMHP.S469324Volume 17(1859-1873)Online publication date: Jul-2024
https://doi.org/10.2147/RMHP.S469324
Lim CPrendez DBoyle LRajivan P(2024)The Impact of Cybersecurity Attacks on Human Trust in Autonomous Vehicle OperationsHuman Factors: The Journal of the Human Factors and Ergonomics Society10.1177/00187208241283321Online publication date: 18-Sep-2024
https://doi.org/10.1177/00187208241283321
Arn LHuang E(2024)“Robots Can Do Disgusting Things, but Also Good Things”: Fostering Children’s Understanding of AI through StorytellingACM Transactions on Computing Education10.1145/367761324:3(1-55)Online publication date: 27-Sep-2024
https://dl.acm.org/doi/10.1145/3677613
Pérez-Messina ICeneda DMiksch S(2024)Enhancing Visual Analytics systems with guidance: A task-driven methodologyComputers & Graphics10.1016/j.cag.2024.104121125(104121)Online publication date: Dec-2024
https://doi.org/10.1016/j.cag.2024.104121
Zhang THe X(2024)Enhancing Explainability in Medical AI: Developing Human-Centered Participatory Design CardsHCI International 2024 – Late Breaking Papers10.1007/978-3-031-76827-9_10(164-194)Online publication date: 31-Dec-2024
https://doi.org/10.1007/978-3-031-76827-9_10
Gardner DTanenbaum T(2023)The access control double bind: How everyday interfaces regulate access and privacy, enable surveillance, and enforce identityConvergence: The International Journal of Research into New Media Technologies10.1177/1354856523119370630:3(1186-1218)Online publication date: 19-Aug-2023
https://doi.org/10.1177/13548565231193706
Pérez-Messina ICeneda DMiksch S(2023)Guided Visual Analytics for Image Selection in Time and SpaceIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2023.332657230:1(66-75)Online publication date: 3-Nov-2023
https://dl.acm.org/doi/10.1109/TVCG.2023.3326572

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Full Text

View this article in Full Text.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View full text|Download PDF

View Issue’s Table of Contents