Abstract
Appropriate reliance on artificial intelligence (AI)-based systems is paramount to leverage increasing AI performance. However, multi-party settings, where multiple parties with diverging interests interact with the support of AI systems, are currently neglected. In this study, we use Heider’s balance theory to derive a framework that allows us to conceptualize and analyze reliance on AI in multi-party settings. We then use this framework to analyze two large design science research projects. First, we analyze financial advisory service encounters, where the role inequality of advisor and client can lead to a dominance of the advisor. Second, we analyze used car market negotiations, where the problem of partial reliance on AI systems creates a misalignment between the two parties, ultimately failing the negotiations. Finally, we discuss implications and future research on AI reliance in multi-party settings and highlight that this study should serve as a starting point in investigating AI reliance in multi-party settings.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
OpenAI: Introducing ChatGPT (2023). https://openai.com/blog/chatgpt
Faisal, A., Kamruzzaman, M., Yigitcanlar, T., Currie, G.: Understanding autonomous vehicles. J. Transp. Land Use 12, 45–72 (2019)
Rädsch, T., Eckhardt, S., Leiser, F., Pandl, K.D., Thiebes, S., Sunyaev, A.: What your radiologist might be missing: using machine learning to identify mislabeled instances of X-ray images (2021)
EU: Ethics guidelines for trustworthy AI | Shaping Europe’s digital future. https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai. Accessed 15 Jan 2024
Chatterjee, S., Sarker, S., Lee, M.J., Xiao, X., Elbanna, A.: A possible conceptualization of the information systems (IS) artifact: a general systems theory perspective 1. Inf. Syst. J. 31, 550–578 (2021)
Sutton, S.G., Arnold, V., Holt, M.: An extension of the theory of technology dominance: capturing the underlying causal complexity. Int. J. Account. Inf. Syst. 50, 100626 (2023)
Lee, J.D., See, K.A.: Trust in automation: designing for appropriate reliance. Hum. Factors 31 (2004)
Schemmer, M., Kuehl, N., Benz, C., Bartos, A., Satzger, G.: Appropriate reliance on ai advice: conceptualization and the effect of explanations. In: Proceedings of the 28th International Conference on Intelligent User Interfaces, pp. 410–422. ACM, Sydney (2023)
Passi, S., Vorvoreanu, M.: Overreliance on AI literature review. Microsoft Research (2022)
Wang, L., Jamieson, G.A., Hollands, J.G.: Selecting methods for the analysis of reliance on automation. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 52, pp. 287–291 (2008)
Dikmen, M., Burns, C.: The effects of domain knowledge on trust in explainable AI and task performance: a case of peer-to-peer lending. Int. J. Hum. Comput. Stud. 162 (2022)
Glick, A., Clayton, M., Angelov, N., Chang, J.: Impact of explainable artificial intelligence assistance on clinical decision-making of novice dental clinicians. JAMIA Open 5 (2022)
Eisenhardt, K.M.: Agency theory: an assessment and review. Acad. Manag. Rev. 14, 57–74 (1989)
Scott, A., Vick, S.: Patients, doctors and contracts: an application of principal-agent theory to the doctor-patient relationship. Scott. J. Polit. Econ. 46, 111–134 (1999)
Golec, J.H.: Empirical tests of a principal-agent model of the investor-investment advisor relationship. J. Financ. Quant. Anal. 27, 81–95 (1992)
Miller, G.J., Whitford, A.B.: Trust and incentives in principal-agent negotiations: the ‘insurance/incentive trade-off.’ J. Theor. Polit. 14, 231–267 (2002)
Chiang, C.-W., Lu, Z., Li, Z., Yin, M.: Are two heads better than one in AI-assisted decision making? Comparing the behavior and performance of groups and individuals in human-AI collaborative recidivism risk assessment. In: Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems, pp. 1–18. ACM, Hamburg (2023)
Heider, F.: Attitudes and cognitive organization. J. Psychol. 21, 107–112 (1946)
Dzindolet, M.T., Pierce, L.G., Beck, H.P., Dawe, L.A.: Misuse and disuse of automated aids. Presented at the Proceedings of the Human Factors and Ergonomics Society Annual Meeting (1999)
Parasuraman, R., Riley, V.: Humans and automation: use, misuse, disuse, abuse. Hum. Factors 39, 230–253 (1997)
Chiang, C.-W., Yin, M.: Exploring the effects of machine learning literacy interventions on laypeople’s reliance on machine learning models. In: International Conference on Intelligent User Interfaces Proceedings, IUI, pp. 148–161. Association for Computing Machinery (2022)
Haight, J.M., Kecojevic, V.: Automation vs. human intervention: what is the best fit for the best performance? Process Saf. Progr. 24, 45–51 (2005)
Nourani, M., et al.: Anchoring bias affects mental model formation and user reliance in explainable AI systems. In: 26th International Conference on Intelligent User Interfaces, pp. 340–350. ACM, College Station (2021)
Schmitt, A., Wambsganss, T., Söllner, M., Janson, A.: Towards a trust reliance paradox? Exploring the gap between perceived trust in and reliance on algorithmic advice. In: International Conference on Information Systems (ICIS), Austin, Texas (2021)
Lai, V., Chen, C., Liao, Q.V., Smith-Renner, A., Tan, C.: Towards a science of human-ai decision making: a survey of empirical studies. arXiv preprint arXiv:2112.11471 (2021)
Guggenberger, T., Lämmermann, L., Urbach, N., Walter, A., Hofmann, P.: Task delegation from AI to humans: a principal-agent perspective. In: ICIS 2023 Proceedings (2023)
Vössing, M., Kühl, N., Lind, M., Satzger, G.: Designing transparency for effective human-ai collaboration. Inf. Syst. Front. 24, 877–895 (2022)
Heider, F.: The Psychology of Interpersonal Relations. Psychology Press (2013)
Aronson, E.: The theory of cognitive dissonance: a current perspective. In: Berkowitz, L. (ed.) Advances in Experimental Social Psychology, pp. 1–34. Academic Press (1969)
Cartwright, D., Harary, F.: Structural balance: a generalization of Heider’s theory. Psychol. Rev. 63, 277–293 (1956)
Roy, S., Gammoh, B.S., Koh, A.C.: Predicting the effectiveness of celebrity endorsements using the balance theory. J. Cust. Behav. 11, 33–52 (2012)
Fink, J.S., Parker, H.M., Brett, M., Higgins, J.: Off-field behavior of athletes and team identification: using social identity theory and balance theory to explain fan reactions. J. Sport Manag. 23, 142–155 (2009)
Min, J.H.J., Chang, H.J.J., Jai, T.-M.C., Ziegler, M.: The effects of celebrity-brand congruence and publicity on consumer attitudes and buying behavior. Fash Text. 6, 10 (2019)
Qi, L., et al.: Structural balance theory-based E-commerce recommendation over big rating data. IEEE Trans. Big Data 4, 301–312 (2018)
Nakanishi, H., Nakazawa, S., Ishida, T., Takanashi, K., Isbister, K.: Can software agents influence human relations? Balance theory in agent-mediated communities. In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 717–724. Association for Computing Machinery, New York (2003)
Bucher, A., Dolata, M., Eckhardt, S., Staehelin, D., Schwabe, G.: Talking to multi-party conversational agents in advisory services: command-based vs. conversational interactions. Proc. ACM Hum.-Comput. Interact. 8, 1–25 (2024)
Eckhardt, S., et al.: “Garbage in, garbage out”: mitigating human biases in data entry by means of artificial intelligence. Presented at the IFIP Conference on Human-Computer Interaction (2023)
Eckhardt, S., Bucher, A., Kalunder, M., Dolata, M., Agotai, D., Schwabe, G.: Secondary mental models: introducing conversational agents in financial advisory service encounters. In: ICIS 2023 Proceedings (2023)
Gregory, R., Muntermann, J.: Theorizing in design science research: inductive versus deductive approaches (2011)
Van de Ven, A.H.: Engaged Scholarship: A Guide for Organizational and Social Research. Oxford University Press (2007)
Eckhardt, S., Sprenkamp, K., Zavolokina, L., Bauer, I., Schwabe, G.: Can artificial intelligence help used-car dealers survive in a data-driven used-car market? In: Drechsler, A., Gerber, A., Hevner, A. (eds.) DESRIST 2022. LNCS, vol. 13229, pp. 115–127. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-06516-3_9
Sonnenberg, C., vom Brocke, J.: Evaluations in the science of the artificial – reconsidering the build-evaluate pattern in design science research. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds.) DESRIST 2012. LNCS, vol. 7286, pp. 381–397. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29863-9_28
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Eckhardt, S., Dolata, M., Bauer-Hänsel, I., Schwabe, G. (2024). Conceptualizing Multi-party AI Reliance for Design Research. In: Mandviwalla, M., Söllner, M., Tuunanen, T. (eds) Design Science Research for a Resilient Future. DESRIST 2024. Lecture Notes in Computer Science, vol 14621. Springer, Cham. https://doi.org/10.1007/978-3-031-61175-9_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-61175-9_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-61174-2
Online ISBN: 978-3-031-61175-9
eBook Packages: Computer ScienceComputer Science (R0)