Philipp Kulms
ABSTRACT
Computer agents are increasingly endowed with anthropomorphic characteristics and autonomous behavior to improve their capabilities for problem-solving and make interactions with humans more natural. This poses new challenges for human users who need to make trust-based decisions in dynamic and complex environments. It remains unclear if people trust agents like other humans and thus apply the same social rules to human--computer interaction (HCI), or rather, if interactions with computers are characterized by idiosyncratic attributions and responses. To this ongoing and crucial debate we contribute an experiment on the impact of anthropomorphic cues on trust and trust-related attributions in a cooperative human--agent setting, permitting the investigation of interdependent, continued, and coordinated decision-making toward a joint goal. Our results reveal an incongruence between self-reported and behavioral trust measures. First, the varying degree of agent anthropomorphism (computer vs. virtual vs. human agent) did not affect people's decision to behaviorally trust the agent by adopting task-specific advice. Behavioral trust was affected by advice quality only. Second, subjective ratings indicate that anthropomorphism did increase self-reported trust.
- D. Antos, Celso de Melo, Jonathan Gratch, and B. Grosz. 2011. The influence of emotion expression on perceptions of trustworthiness in negotiation. In Proceedings of the 25th AAAI Conference. AAAI Press, Menlo Park, CA, 772--778. Google Scholar
Digital Library
- R. Axelrod. 1984. The evolution of cooperation. Basic Books, New York.Google Scholar
- Daniel Balliet and Paul AM Van Lange. 2013. Trust, conflict, and cooperation: A meta-analysis. Psychological Bulletin 139, 5 (2013), 1090.Google ScholarCross Ref
- Kirsten Bergmann, Friederike Eyssel, and Stefan Kopp. 2012. A Second Chance to Make a First Impression? How Appearance and Nonverbal Behavior Affect Perceived Warmth and Competence of Virtual Agents over Time. In Intelligent Virtual Agents. Springer, Berlin, Heidelberg, 126--138. Google ScholarDigital Library
- Timothy W. Bickmore and Rosalind W. Picard. 2005. Establishing and maintaining long-term human-computer relationships. ACM Transactions on Computer-Human Interaction 12, 2 (2005), 293--327. Google ScholarDigital Library
- Silvia Bonaccio and Reeshad S Dalal. 2006. Advice taking and decision-making: An integrative literature review, and implications for the organizational sciences. Organizational Behavior and Human Decision Processes 101, 2 (2006), 127--151.Google ScholarCross Ref
- R Thomas Boone and Ross Buck. 2003. Emotional expressivity and trustworthiness: The role of nonverbal behavior in the evolution of cooperation. Journal of Nonverbal Behavior 27, 3 (2003), 163--182.Google ScholarCross Ref
- Jeffrey M Bradshaw, Virginia Dignum, Catholijn Jonker, and Maarten Sierhuis. 2012. Human--agent--robot teamwork. Intelligent Systems, IEEE 27, 2 (2012), 8--13. Google ScholarDigital Library
- Cynthia Breazeal, Paul L. Harris, David DeSteno, Jacqueline M. Kory Westlund, Leah Dickens, and Sooyeon Jeong. 2016. Young Children Treat Robots as Informants. Topics in Cognitive Science 8, 2 (2016), 481--491.Google ScholarCross Ref
- Cynthia Breazeal, C. D. Kidd, A. L. Thomaz, G. Hoffman, and M. Berlin. 2005. Effects of nonverbal communication on efficiency and robustness in human-robot teamwork. In 2005 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 708--713.Google Scholar
- Gordon Briggs and Matthias Scheutz. 2015. "Sorry, I can't do that": Developing mechanisms to appropriately reject directives in human-robot interactions. In 2015 AAAI Fall Symposium Series.Google Scholar
- Elizabeth Broadbent, Vinayak Kumar, Xingyan Li, John Sollers 3rd, Rebecca Q Stafford, Bruce A MacDonald, and Daniel M Wegner. 2013. Robots with display screens: A robot with a more humanlike face display is perceived to have more mind and a better personality. PLOS ONE 8, 8 (2013), e72589.Google ScholarCross Ref
- Sarah F. Brosnan, Lucie Salwiczek, and Redouan Bshary. 2010. The interplay of cognition and cooperation. Philosophical transactions of the Royal Society of London. Series B, Biological sciences 365, 1553 (2010), 2699--2710.Google Scholar
- Victoria Buchholz, Philipp Kulms, and Stefan Kopp. 2017. It's (Not) Your Fault! Blame and Trust Repair in Human-Agent Cooperation. Kognitive Systeme 2017, 1 (2017).Google Scholar
- Colin F Camerer. 2003. Behavioral game theory: Experiments in strategic interaction. Princeton University Press.Google Scholar
- Justine Cassell and Timothy Bickmore. 2000. External manifestations of trustworthiness in the interface. Commun. ACM 43, 12 (2000), 50--56. Google ScholarDigital Library
- Ahyoung Choi, Celso M. de Melo, Peter Khooshabeh, Woontack Woo, and Jonathan Gratch. 2015. Physiological evidence for a dual process model of the social effects of emotion in computers. International Journal of Human-Computer Studies 74 (2015), 41--53. Google ScholarDigital Library
- Jason A. Colquitt, Brent A. Scott, and Jeffery A. LePine. 2007. Trust, trustworthiness, and trust propensity: A meta-analytic test of their unique relationships with risk taking and job performance. Journal of Applied Psychology 92, 4 (2007), 909--927.Google ScholarCross Ref
- Kerstin Dautenhahn. 1998. The art of designing socially intelligent agents: Science, fiction, and the human in the loop. Applied Artificial Intelligence 12, 7-8 (1998), 573--617.Google ScholarCross Ref
- R. M. Dawes. 1980. Social Dilemmas. Annual Review of Psychology 31, 1 (1980), 169--193.Google ScholarCross Ref
- Iwan de Kok, Julian Hough, Felix Hülsmann, Mario Botsch, David Schlangen, and Stefan Kopp. 2015. A Multimodal System for Real-Time Action Instruction in Motor Skill Learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction. ACM, 355--362. Google ScholarDigital Library
- Celso M. de Melo, Peter J. Carnevale, Stephen J. Read, and Jonathan Gratch. 2014. Reading people's minds from emotion expressions in interdependent decision making. Journal of Personality and Social Psychology 106, 1 (2014), 73--88.Google ScholarCross Ref
- Celso M. de Melo and Jonathan Gratch. 2015. Beyond Believability: Quantifying the Differences Between Real and Virtual Humans. In Intelligent Virtual Agents. Vol. LNCS 9238. Springer, Berlin, Heidelberg, 109--118.Google Scholar
- Celso M. de Melo, Jonathan Gratch, and Peter J Carnevale. 2015. Humans versus Computers: Impact of Emotion Expressions on People's Decision Making. IEEE Transactions on Affective Computing 6, 2 (2015), 127--136.Google ScholarDigital Library
- Ewart J de Visser, Samuel S. Monfort, Kimberly Goodyear, Li Lu, Martin O'Hara, Mary R Lee, Raja Parasuraman, and Frank Krueger. 2017. A little anthropomorphism goes a long way: Effects of oxytocin on trust, compliance, and team performance with automated agents. Human Factors 59, 1 (2017), 116--133.Google ScholarCross Ref
- Ewart J de Visser, Samuel S. Monfort, R. McKendrick, M. A. Smith, P. E. McKnight, F. Krueger, and R. Parasuraman. 2016. Almost human: Anthropomorphism increases trust resilience in cognitive agents. Journal of Experimental Psychology: Applied 22, 3 (2016), 331--349.Google ScholarCross Ref
- David DeSteno, Cynthia Breazeal, Robert H Frank, David Pizarro, Jolie Baumann, Leah Dickens, and Jin Joo Lee. 2012. Detecting the trustworthiness of novel partners in economic exchange. Psychological Science (2012), 1549--1556.Google Scholar
- David DeVault, Ron Artstein, Grace Benn, Teresa Dey, Ed Fast, Alesia Gainer, Kallirroi Georgila, Jon Gratch, Arno Hartholt, Margaux Lhommet, et al. 2014. SimSensei Kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of the 2014 International Conference on Autonomous Agents and Multi-Agent Systems. 1061--1068. Google ScholarDigital Library
- Jaap J Dijkstra, Wim BG Liebrand, and Ellen Timminga. 1998. Persuasiveness of expert systems. Behaviour & Information Technology 17, 3 (1998), 155--163.Google ScholarCross Ref
- Mary T Dzindolet, Scott A Peterson, Regina A Pomranky, Linda G Pierce, and Hall P Beck. 2003. The role of trust in automation reliance. International Journal of Human-Computer Studies 58, 6 (2003), 697--718. Google ScholarDigital Library
- Susan T. Fiske, Amy J. C. Cuddy, and Peter Glick. 2007. Universal dimensions of social cognition: warmth and competence. Trends in Cognitive Sciences 11, 2 (2007), 77--83.Google ScholarCross Ref
- BJ Fogg and Hsiang Tseng. 1999. The elements of computer credibility. In Proceedings of the SIGCHI conference on Human Factors in Computing Systems. ACM, 80--87. Google ScholarDigital Library
- Simon Gächter. 2004. Behavioral game theory. In Blackwell Handbook of Judgment and Decision Making, D.J. Koehler and Harvey Nigel (Eds.). Blackwell: Malden, Massachusetts, 485--503.Google Scholar
- Jonathan Gratch, David DeVault, and Gale Lucas. 2016. The Benefits of Virtual Humans for Teaching Negotiation. In Intelligent Virtual Agents. Springer International Publishing, 283--294.Google Scholar
- Jonathan Gratch and Stacy Marsella. 2004. A domain-independent framework for modeling emotion. Cognitive Systems Research 5, 4 (2004), 269--306. Google ScholarDigital Library
- Jonathan Gratch, Ning Wang, Jillian Gerten, Edward Fast, and Robin Duffy. 2007. Creating rapport with virtual agents. In International Conference on Intelligent Virtual Agents. Springer, 125--138. Google ScholarDigital Library
- P. A. Hancock, D. R. Billings, K. E. Schaefer, Chen, J. Y. C., de Visser, E. J., and R. Parasuraman. 2011. A Meta-Analysis of Factors Affecting Trust in Human-Robot Interaction. Human Factors: The Journal of the Human Factors and Ergonomics Society 53, 5 (2011), 517--527.Google ScholarCross Ref
- Robert R Hoffman, Matthew Johnson, Jeffrey M Bradshaw, and Al Underbrink. 2013. Trust in automation. IEEE Intelligent Systems 28, 1 (2013), 84--88. Google ScholarDigital Library
- Katherine Isbister and Clifford Nass. 2000. Consistency of personality in interactive characters: verbal cues, non-verbal cues, and user characteristics. International Journal of Human-Computer Studies 53, 2 (2000), 251--267. Google ScholarDigital Library
- G. R. Jones and J. M. George. 1998. The experience and evolution of trust: Implications for cooperation and teamwork. Academy of Management Review 23, 3 (1998), 531--546.Google ScholarCross Ref
- Jillian J. Jordan, Moshe Hoffman, Martin A. Nowak, and David G. Rand. 2016. Uncalculating cooperation is used to signal trustworthiness. (2016).Google Scholar
- Charles M. Judd, Laurie James-Hawkins, Vincent Yzerbyt, and Yoshihisa Kashima. 2005. Fundamental dimensions of social judgment: understanding the relations between judgments of competence and warmth. Journal of Personality and Social Psychology 89, 6 (2005), 899--913.Google ScholarCross Ref
- Jari Kätsyri, Klaus Förger, Meeri Mäkäräinen, and Tapio Takala. 2015. A review of empirical evidence on different uncanny valley hypotheses: Support for perceptual mismatch as one road to the valley of eeriness. Frontiers in Psychology 6 (2015), 390.Google ScholarCross Ref
- Sara Kiesler, Lee Sproull, and Keith Waters. 1996. A prisoner's dilemma experiment on cooperation with people and human-like computers. Journal of Personality and Social Psychology 70, 1 (1996), 47.Google ScholarCross Ref
- Stefan Kopp, Brigitte Krenn, Stacy Marsella, Andrew N Marshall, Catherine Pelachaud, Hannes Pirker, Kristinn R Thórisson, and Hannes Vilhjálmsson. 2006. Towards a common framework for multimodal generation: The behavior markup language. In International Conference on Intelligent Virtual Agents. Springer, 205--217. Google ScholarDigital Library
- N. C. Krämer, A. M. Rosenthal-von der Pütten, and L. Hoffmann. 2015. Social Effects of Virtual and Robot Companions. In The Handbook of the Psychology of Communication Technology, S. S. Sundar (Ed.). John Wiley & Sons, 137--159.Google Scholar
- Philipp Kulms and Stefan Kopp. 2016. The Effect of Embodiment and Competence on Trust and Cooperation in Human--Agent Interaction. In International Conference on Intelligent Virtual Agents. Springer, 75--84.Google Scholar
- Philipp Kulms and Stefan Kopp. 2018. A social cognition perspective on human--computer trust: The effect of perceived warmth and competence on trust in decision-making with computers. Frontiers in Digital Humanities 5 (2018), 14.Google ScholarCross Ref
- Ellen J Langer. 1989. Mindfulness. Addison-Wesley/Addison Wesley Longman.Google Scholar
- John Lee and Neville Moray. 1992. Trust, control strategies and allocation of function in human-machine systems. Ergonomics 35, 10 (1992), 1243--1270.Google ScholarCross Ref
- J. D. Lee and K. A. See. 2004. Trust in Automation: Designing for Appropriate Reliance. Human Factors: The Journal of the Human Factors and Ergonomics Society 46, 1 (2004), 50--80.Google ScholarCross Ref
- Raz Lin and Sarit Kraus. 2010. Can automated agents proficiently negotiate with humans? Commun. ACM 53, 1 (2010), 78--88. Google ScholarDigital Library
- Gale Lucas, Giota Stratou, Shari Lieblich, and Jonathan Gratch. 2016. Trust me: multimodal signals of trustworthiness. In Proceedings of the 18th ACM International Conference on Multimodal Interaction. 5--12. Google ScholarDigital Library
- Gale M Lucas, Jonathan Gratch, Aisha King, and Louis-Philippe Morency. 2014. ItâĂŹs only a computer: Virtual humans increase willingness to disclose. Computers in Human Behavior 37 (2014), 94--100. Google ScholarDigital Library
- P. Madhavan and D. A. Wiegmann. 2007. Similarities and differences between human--human and human--automation trust: An integrative review. Theoretical Issues in Ergonomics Science 8, 4 (2007), 277--301.Google ScholarCross Ref
- Bertram F. Malle and Matthias Scheutz. 2014. Moral competence in social robots. In 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering. IEEE, 1--6. Google ScholarDigital Library
- Roger C. Mayer, James H. Davis, and F. David Schoorman. 1995. An Integrative Model of Organizational Trust. The Academy of Management Review 20, 3 (1995), 709--734.Google ScholarCross Ref
- Daniel J. McAllister. 1995. Affect- and Cognition-Based Trust as Foundations for Interpersonal Cooperation in Organizations. The Academy of Management Journal 38, 1 (1995), 24--59.Google ScholarCross Ref
- K. Miwa, H. Terai, and S. Hirose. 2008. Social Responses to Collaborator: Dilemma Game with Human and Computer Agent. In Proceedings of the 30th Annual Conference of the Cognitive Science Society, B. C. Love, K. McRae, and V. M. Sloutsky (Eds.). Cognitive Science Society, Austin, TX, 2455--2460.Google Scholar
- Bonnie M. Muir. 1987. Trust between humans and machines, and the design of decision aids. International Journal of Man-Machine Studies 27, 5-6 (1987), 527--539. Google ScholarDigital Library
- Clifford Nass, BJ Fogg, and Youngme Moon. 1996. Can computers be teammates? International Journal of Human-Computer Studies 45, 6 (1996), 669--678. Google ScholarDigital Library
- Clifford Nass and Youngme Moon. 2000. Machines and Mindlessness: Social Responses to Computers. Journal of Social Issues 56, 1 (2000), 81--103.Google ScholarCross Ref
- Clifford Nass, Jonathan Steuer, and Ellen R. Tauber. 1994. Computers are social actors. In CHI '94 Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, New York, 72--78. Google ScholarDigital Library
- Richard Pak, Nicole Fink, Margaux Price, Brock Bass, and Lindsay Sturre. 2012. Decision support aids with anthropomorphic characteristics influence trust and performance in younger and older adults. Ergonomics 55, 9 (2012), 1059--1072.Google ScholarCross Ref
- Raja Parasuraman and Dietrich H Manzey. 2010. Complacency and bias in human use of automation: An attentional integration. Human Factors 52, 3 (2010), 381--410.Google ScholarCross Ref
- Raja Parasuraman and Victor Riley. 1997. Humans and Automation: Use, Misuse, Disuse, Abuse. Human Factors 39, 2 (1997), 230--253.Google ScholarCross Ref
- Byron Reeves and Clifford Nass. 1996. The Media Equation: How people treat computers, television, and new media like real people and places. Cambridge University Press, Cambridge, UK. Google ScholarDigital Library
- John K. Rempel, John G. Holmes, and Mark P. Zanna. 1985. Trust in close relationships. Journal of Personality and Social Psychology 49, 1 (1985), 95--112.Google ScholarCross Ref
- Eduardo Benítez Sandoval, Jürgen Brandstetter, Mohammad Obaid, and Christoph Bartneck. 2015. Reciprocity in Human-Robot Interaction: A Quantitative Approach Through the Prisoner's Dilemma and the Ultimatum Game. International Journal of Social Robotics (2015).Google Scholar
- Thomas B Sheridan and Raja Parasuraman. 2005. Human-automation interaction. Reviews of human factors and ergonomics 1, 1 (2005), 89--129.Google Scholar
- Candace L. Sidner, Christopher Lee, Cory D. Kidd, Neal Lesh, and Charles Rich. 2005. Explorations in engagement for humans and robots. Artificial Intelligence 166, 1 (2005), 140--164. Google ScholarCross Ref
- Alexander Todorov. 2008. Evaluating faces on trustworthiness. Annals of the New York Academy of Sciences 1124, 1 (2008), 208--224.Google ScholarCross Ref
- Kees Van Dongen and Peter-Paul Van Maanen. 2013. A framework for explaining reliance on decision aids. International Journal of Human-Computer Studies 71, 4 (2013), 410--424. Google ScholarDigital Library
- A. van Wissen, Y. Gal, B. A. Kamphorst, and M. V. Dignum. 2012. Human--agent teamwork in dynamic environments. Computers in Human Behavior 28, 1 (2012), 23--33. Google ScholarDigital Library
- Anna-Lisa Vollmer, Manuel Mühlig, Jochen J Steil, Karola Pitsch, Jannik Fritsch, Katharina J Rohlfing, and Britta Wrede. 2014. Robots show us how to teach them: Feedback from robots shapes tutoring behavior during action learning. PlOS ONE 9, 3 (2014), e91349.Google ScholarCross Ref
- Steffen Walter, Cornelia Wendt, Jan Böhnke, Stephen Crawcour, Jun-Wen Tan, Andre Chan, Kerstin Limbrecht, Sascha Gruss, and Harald C Traue. 2014. Similarities and differences of emotions in human--machine and human--human interactions: What kind of emotions are relevant for future companion systems? Ergonomics 57, 3 (2014), 374--386.Google ScholarCross Ref
- Weiquan Wang and Izak Benbasat. 2008. Attributions of trust in decision support technologies: A study of recommendation agents for e-commerce. Journal of Management Information Systems 24, 4 (2008), 249--273. Google ScholarDigital Library
- Adam Waytz, Joy Heafner, and Nicholas Epley. 2014. The mind in the machine: Anthropomorphism increases trust in an autonomous vehicle. Journal of Experimental Social Psychology 52 (2014), 113--117.Google ScholarCross Ref
- Joel S Winston, Bryan A Strange, John O'Doherty, and Raymond J Dolan. 2002. Automatic and intentional brain responses during evaluation of trustworthiness of faces. Nature Neuroscience 5, 3 (2002), 277--283.Google ScholarCross Ref
- Ramin Yaghoubzadeh, Marcel Kramer, Karola Pitsch, and Stefan Kopp. 2013. Virtual agents as daily assistants for elderly or cognitively impaired people. In Intelligent Virtual Agents. Springer International Publishing, 79--91.Google Scholar
Index Terms
- More Human-Likeness, More Trust?: The Effect of Anthropomorphism on Self-Reported and Behavioral Trust in Continued and Interdependent Human-Agent Cooperation
Recommendations
Trust and Anthropomorphism in Tandem: The Interrelated Nature of Automated Agent Appearance and Reliability in Trustworthiness Perceptions
DIS '21: Proceedings of the 2021 ACM Designing Interactive Systems ConferenceAnthropomorphism in the design of interface agents is implicitly linked to increasing user trust and acceptance. However, the role of perceived anthropomorphism and perceived trustworthiness in trust appropriateness given a system’s capabilities and ...
Trust in an AI versus a Human teammate: The effects of teammate identity and performance on Human-AI cooperation
AbstractRecent advances in artificial intelligence (AI) enable researchers to create more powerful AI agents that are becoming competent teammates for humans. However, human distrust of AI is a critical factor that may impede human-AI ...
Highlights- Humans accept their AI teammate's decision less often when they are deceived about the identity of the AI as another human.
Comparing Human Trust Attitudes Towards Human and Agent Teammates
HAI '20: Proceedings of the 8th International Conference on Human-Agent InteractionAgents' roles in our lives increasingly matter as they engage with people in a variety of important tasks. To achieve successful human-agent teamwork, it is critical to know the differences and similarities in people's attitudes towards human and agent ...
Comments