research-article

Do people hold a humanoid robot morally accountable for the harm it causes?

Authors:

Peter H. Kahn, Jr.,

Takayuki Kanda,

Hiroshi Ishiguro,

Jolina H. Ruckert,

Heather E. Gary,

Aimee L. Reichert,

Nathan G. Freier,

Rachel L. SeversonAuthors Info & Claims

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

Pages 33 - 40

https://doi.org/10.1145/2157689.2157696

Published: 05 March 2012 Publication History

Abstract

Robots will increasingly take on roles in our social lives where they can cause humans harm. When robots do so, will people hold robots morally accountable? To investigate this question, 40 undergraduate students individually engaged in a 15-minute interaction with ATR's humanoid robot, Robovie. The interaction culminated in a situation where Robovie incorrectly assessed the participant's performance in a game, and prevented the participant from winning a $20 prize. Each participant was then interviewed in a 50-minute session. Results showed that all of the participants engaged socially with Robovie, and many of them conceptualized Robovie as having mental/emotional and social attributes. Sixty-five percent of the participants attributed some level of moral accountability to Robovie. Statistically, participants held Robovie less accountable than they would a human, but more accountable than they would a vending machine. Results are discussed in terms of the New Ontological Category Hypothesis and robotic warfare.

References

[1]

Searle, J. R. (1992). The rediscovery of the mind. Cambridge, MA: MIT Press.

Digital Library

[2]

Shen, S. (2011). The curious case of human-robot morality. Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction, 249--250.

Digital Library

[3]

Powers, T. M. (2011). Incremental machine ethics. IEEE Robotics & Automation Magazine, 18, 51--58.

[4]

Dennett, D. C. (1998). When HAL kills, who's to blame? Computer ethics. In D. G. Stork (Ed.), HAL's Legacy: 2001's Computer as Dream and Reality (pp. 351--365). Cambridge, MA: MIT Press.

[5]

Asaro, P. M. (in press). A body to kick, but still no soul to damn: Legal perspectives on robotics. In P. Lin, K. Abney, & G. Bekey (Eds.), Robot ethics: The ethical and social implications of robotics. Cambridge, MA: MIT Press.

[6]

Scheffler, S. (1992). Human morality. New York, NY: Oxford University Press.

[7]

Dworkin, R. (1986). Law's empire. Cambridge, MA: Harvard University Press.

[8]

Friedman, B., & Millet, L. (1995). "It's the computer's fault": Reasoning about computers as moral agents. Proceedings of the Conference on Human Factors in Computing Systems, 226--227. New York, NY: ACM Press.

Digital Library

[9]

Fong, T., Nourbakhsh, I., & Dautenhahn, K. (2003). A survey of socially interactive robots. Robotics and Autonomous Systems, 42, 143--166.

[10]

Iwamura, Y., Shiomi, M., Kanda, T., Ishiguro, H., & Hagita, N. (2011). Do elderly people prefer a conversational humanoid as a shopping assistant partner in supermarkets? Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction, 449--456.

Digital Library

[11]

Kanda, T., Hirano, T., Eaton, D., & Ishiguro, H. (2004). Interactive robots as social partners and peer tutors for children. Human Computer Interaction, 19, 61--84.

Digital Library

[12]

Tanaka, F., Cicourel, A., & Movellan, J. R. (2007). Socialization between toddlers and robots at an early childhood education center. Proceedings of the National Academy of Sciences of the Untied States of America, 104, 17954--17958.

[13]

Short, E., Hart, J., Vu, M., & Scassellati, B. (2010). No fair!! An interaction with a cheating robot. Proceedings of the 5th ACM/IEEE International Conference on Human-Robot Interaction, 219--226.

Digital Library

[14]

Kahn, P. H. Jr., Freier, N. G., Kanda, T., Ishiguro, H., Ruckert, J. H., Severson, R., & Kane, S. K. (2008). Design patterns for sociality in human-robot interaction. Proceedings of the 3rd ACM/IEEE International Conference on Human-Robot Interaction, 97--104.

Digital Library

[15]

Turiel, E. (1998). The development of morality. In W. Damon (Editor-in-Chief), Handbook of Child Psychology: Vol. 3. Social, emotional, and personality development (pp. 863--932). New York, NY: Wiley.

[16]

Kahn, P. H., Jr., Kanda, T., Ishiguro, H., Ruckert, J. H., Severson, R. L., Freier, N. G., & Reichert, A. L. (2010). Coding manual for the "Robovie, you need to go into the closet now!" study. Retrieved from University of Washington, ResearchWorks Archive: https://digital.lib.washington.edu/xmlui/handle/1773/15887

[17]

Kahn, P. H., Jr., Kanda, T., Ishiguro, H., Freier, N. G., Severson, R. L., Gill, B. T., & Shen, S. (in press). "Robovie, you'll have to go into the closet now": Children's social and moral relationships with a humanoid robot. Developmental Psychology.

[18]

Kahn, P. H., Jr., Reichert, A. L., Gary, H. E., Kanda, T., Ishiguro, H., Shen, S., & Gill, B. T. (2011). The new ontological category hypothesis in human-robot interaction. Proceedings of the 6th ACM/IEEE International Conference on Human-Robot Interaction, 159--160.

Digital Library

[19]

Singer, P. W. (2009). Wired for war: The robotics revolution and conflict in the 21st century. New York, NY: Penguin.

Cited By

Georgiou NFlanagan TScassellati BKushnir TJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Perceived Morality of Robot and Human Transgressors Varies By Perceived Ability to FeelProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721600(919-928)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721600
Girouard-Hallam LDanovitch J(2025)Children's trust in Google's ability to answer questions about the past, present, and futureComputers in Human Behavior10.1016/j.chb.2024.108496165(108496)Online publication date: Apr-2025
https://doi.org/10.1016/j.chb.2024.108496
Gigandet RDiana MOuadada KNazir T(2024)Beyond Explicit Acknowledgment: Brain Response Evidence of Human Skepticism towards Robotic EmotionsRobotics10.3390/robotics1305006713:5(67)Online publication date: 28-Apr-2024
https://doi.org/10.3390/robotics13050067
Show More Cited By

Index Terms

Do people hold a humanoid robot morally accountable for the harm it causes?
1. Social and professional topics

Recommendations

People May Punish, But Not Blame Robots
CHI '21: Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems

As robots may take a greater part in our moral decision-making processes, whether people hold them accountable for moral harm becomes critical to explore. Blame and punishment signify moral accountability, often involving emotions. We quantitatively ...
Will People Keep the Secret of a Humanoid Robot?: Psychological Intimacy in HRI
HRI '15: Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction

Will people keep the secret of a socially compelling robot who shares, in confidence, a "personal" (robot) failing? Toward answering this question, 81 adults participated in a 20-minute interaction with (a) a humanoid robot (Robovie) interacting in a ...
Differences between Young and Old Users when Interacting with a Humanoid Robot: A Qualitative Usability Study
HRI '18: Companion of the 2018 ACM/IEEE International Conference on Human-Robot Interaction

There is a growing body of knowledge on how people interact with robots, but limited information on the difference between young and old adults in their preferences when interacting with humanoid robots. Our goals in the current study were: (1) to ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

HRI '12: Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction

March 2012

518 pages

ISBN:9781450310635

DOI:10.1145/2157689

General Chairs:
Holly Yanco
University of Massachusetts Lowell, USA
,
Aaron Steinfeld
Carnegie Mellon University, USA
,
Program Chairs:
Vanessa Evers
University of Amsterdam, The Netherlands
,
Odest Chadwicke Jenkins
Brown University, USA

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

In-Cooperation

IEEE-RAS: Robotics and Automation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 March 2012

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

HRI'12

Sponsor:

HRI'12: International Conference on Human-Robot Interaction

March 5 - 8, 2012

Massachusetts, Boston, USA

Acceptance Rates

Overall Acceptance Rate 268 of 1,124 submissions, 24%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

106
Total Citations
View Citations
1,253
Total Downloads

Downloads (Last 12 months)119
Downloads (Last 6 weeks)15

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Georgiou NFlanagan TScassellati BKushnir TJohal WLemaignan SBrščić DVázquez MCharisi V(2025)Perceived Morality of Robot and Human Transgressors Varies By Perceived Ability to FeelProceedings of the 2025 ACM/IEEE International Conference on Human-Robot Interaction10.5555/3721488.3721600(919-928)Online publication date: 4-Mar-2025
https://dl.acm.org/doi/10.5555/3721488.3721600
Girouard-Hallam LDanovitch J(2025)Children's trust in Google's ability to answer questions about the past, present, and futureComputers in Human Behavior10.1016/j.chb.2024.108496165(108496)Online publication date: Apr-2025
https://doi.org/10.1016/j.chb.2024.108496
Gigandet RDiana MOuadada KNazir T(2024)Beyond Explicit Acknowledgment: Brain Response Evidence of Human Skepticism towards Robotic EmotionsRobotics10.3390/robotics1305006713:5(67)Online publication date: 28-Apr-2024
https://doi.org/10.3390/robotics13050067
Fasoro A(2024)Cultivating Dignity in Intelligent SystemsPhilosophies10.3390/philosophies90200469:2(46)Online publication date: 1-Apr-2024
https://doi.org/10.3390/philosophies9020046
Zhang ZJia MLee HYao BDas SLerner AWang DLi T(2024)“It's a Fair Game”, or Is It? Examining How Users Navigate Disclosure Risks and Benefits When Using LLM-Based Conversational AgentsProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642385(1-26)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642385
Oimann ATollon F(2024)Responsibility Gaps and Technology: Old Wine in New Bottles?Journal of Applied Philosophy10.1111/japp.12763Online publication date: 12-Sep-2024
https://doi.org/10.1111/japp.12763
Mu YKarasawa M(2024)Blame attribution and intentionality perception of human versus robot drivers: Implications for judgments about autonomous vehicles in moral dilemma contextsCogent Psychology10.1080/23311908.2024.238429811:1Online publication date: Aug-2024
https://doi.org/10.1080/23311908.2024.2384298
Shank DDew MSajjad F(2024)Moral behaviour alters impressions of humans and AIs on teams: unethical AIs are more powerful while ethical humans are nicerBehaviour & Information Technology10.1080/0144929X.2024.2403651(1-12)Online publication date: 21-Oct-2024
https://doi.org/10.1080/0144929X.2024.2403651
Simmler M(2024)Ensuring Accountability for Robots and AI under Criminal LawThe Cambridge Handbook of the Law, Policy, and Regulation for Human–Robot Interaction10.1017/9781009386708.050(798-812)Online publication date: 7-Dec-2024
https://doi.org/10.1017/9781009386708.050
Hu Y(2024)Robot Criminal LiabilityThe Cambridge Handbook of the Law, Policy, and Regulation for Human–Robot Interaction10.1017/9781009386708.042(638-655)Online publication date: 7-Dec-2024
https://doi.org/10.1017/9781009386708.042
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten