Abstract
Virtue ethics seems to be a promising moral theory for understanding and interpreting the development and behavior of artificial moral agents. Virtuous artificial agents would blur traditional distinctions between different sorts of moral machines and could make a claim to membership in the moral community. Accordingly, we investigate the “machine question” by studying whether virtue or vice can be attributed to artificial intelligence; that is, are people willing to judge machines as possessing moral character? An experiment describes situations where either human or AI agents engage in virtuous or vicious behavior and experiment participants then judge their level of virtue or vice. The scenarios represent different virtue ethics domains of truth, justice, fear, wealth, and honor. Quantitative and qualitative analyses show that moral attributions are weakened for AIs compared to humans, and the reasoning and explanations for the attributions are varied and more complex. On “relational” views of membership in the moral community, virtuous machines would indeed be included, even if they are indeed weakened. Hence, while our moral relationships with artificial agents may be of the same types, they may yet remain substantively different than our relationships to human beings.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
A fourth type discussed by Moor, “ethical-impact machines,” are not autonomous in the ways that would make them relevant to the discussion here.
The point of the distinction seems designed to mark a gap between machines that simply produce good outcomes or do not wrong us, and machines that approximate moral agency insofar as they are at least causally responsible for the decisions they make, actions they take, and consequences they produce, just as the further distinction between explicit and full moral agents seems intended to forestall intuitions about human (moral) exceptionality. But thinking about explicit ethical agency in terms of explicit ethical reasoning makes serious presumptions about the nature of ethics, ethical reasoning, and moral psychology. After all, it seems that human beings are capable of behaving ethically without anything like the explicit, conscious representation of ethical rules (Hitlin 2008). We seem to be able to respond morally to situations on the go without deliberation in all sorts of situations.
It might be the case that machine learning could allow for the instantiation of a utilitarian or deontological AMA; such an algorithm, properly trained, would act in ways that would be, in fact, the same as one would arrive at by the proper and consistent application of the principle of utility or, say, the Categorical Imperative, respectively. Thanks to an anonymous referee for making explicit this possibility. But insofar as the actual application of the rules is genuinely opaque—say, in the way that the behavior of AlphaGO was opaque—it seems to us that this would be, in appearance, similar enough to the acquisition of the virtues, i.e., responding reliably to salient features of a situation, as to be the same sort of moral agent we are interested in. In any case, we do not intend to claim that a moral machine must look like a virtuous agent, but only to suggest that it seems a plausible candidate, and to draw out some implications.
An analogous worry about the authenticity of care, or feelings in general, on the part of machines, can be found in Turkle (2011).
From a virtue ethics perspective, it might be that the very language of “full moral agents” isn’t quite adequate; it’s clearly beyond the scope of this paper to propose or defend a replacement concept, but it seems like the thrust of the argument of this paper doesn’t depend on these sorts of details.
For an example that sketches how this might be done in the case of environmental ethics, see Gamez (2018).
References
Agar N (2019) How to treat machines that might have minds. Philos Technol. https://doi.org/10.1007/s13347-019-00357-8
Annas J (2016) Learning virtue rules: the issue of thick concepts. In: Annas J, Narvaez D, Snow NE (eds) Developing the virtues: integrating perspectives. Oxford University Press, New York, pp 224–234
Aristotle (2011) Aristotle’s nicomachean ethics (trans and ed: Bartlett RC, Collins SD). University of Chicago, Chicago
Berberich N, Diepold K (2018) The virtuous machine-old ethics for new technology? arXiv preprint arXiv:1806.10322
Bigman YE, Gray K (2018) People are averse to machines making moral decisions. Cognition 181:21–34. https://doi.org/10.1016/j.cognition.2018.08.003
Boden MA (2006) Mind as machine: a history of cognitive science. Clarendon Press, Oxford
Boden MA (2016) AI: its nature and future. Oxford University Press, New York
Bostrom N (2014) Superintelligence: paths, dangers, strategies. Oxford, New York
Coeckelbergh M (2010) Robot rights? Towards a social-relational justification of moral consideration. Ethics Inf Technol 12(3):209–221
Danaher J (2016) The threat of algocracy: reality, resistance, and accommodation. Philos Technol 29:245–268
Danaher J (2019) Welcoming robots into the moral circle: a defence of ethical behaviourism. Sci Eng Ethics. https://doi.org/10.1007/s11948-019-00119-x
Dastur F (2011) The question of the other in French phenomenology. Cont Philos Rev 44(2):165–178
Emerging technology from the arXiv (2015) Why self-driving cars must be programmed to kill. MIT Technology Review. https://www.technologyreview.com. Accessed 22 Oct 2015
Eubanks V (2018) Automating inequality: how high-tech tools profile, police, and punish the poor. St Martin’s, New York
Gamez P (2018) Metaphysics or metaphors for the anthropocene? Scientific naturalism and the agency of things. Open Philos 1(1):191–212
Govindarajulu NS, Bringsjord S, Ghosh R, Sarathy V (2019) Toward the engineering of virtuous machines. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, pp 29–35. ACM
Gray K, Wegner D (2012) Feeling robots and human zombies: mind perception and the uncanny valley. Cognition 125(1):125–130. https://doi.org/10.1016/j.cognition.2012.06.007
Gray K, Young L, Waytz A (2012) Mind perception is the essence of morality. Psychol Inq 23(2):101–124. https://doi.org/10.1080/1047840x.2012.651387
Gunkel DJ (2012) The machine question: critical perspectives on AI, robots, and ethics. MIT Press, Cambridge
Gunkel DJ (2018) Robot rights. MIT Press, Cambridge
Hitlin S (2008) Moral selves, evil selves: the social psychology of conscience. Palgrave Macmillan, New York
Howard D, Muntean I (2016) A minimalist model of the artificial autonomous moral agent (AAMA). In: 2016 AAAI Spring Symposium Series, Palo Alto California, March 2016. https://www.aaai.org/ocs/index.php/SSS/SSS16/paper/view/12760/11954. Accessed 20 Apr 2020
Howard D, Muntean I (2017) Artificial moral cognition: moral functionalism and autonomous moral agency. In: Philos Comput, pp 121–159. Springer, Cham
Hursthouse R (1999) On virtue ethics. OUP, Oxford
Keown D (2005) Buddhist ethics: a very short introduction. Oxford University Press, New York
Knobe J (2003a) Intentional action in folk psychology: an experimental investigation. Philos Psychol 16(2):309–324
Knobe J (2003b) Intentional action and side effects in ordinary language. Analysis 63(3):190–194
Kraut R (2018) Aristotle’s ethics. The Stanford encyclopedia of philosophy. https://plato.stanford.edu/entries/Aristotle-ethics. Accessed 15 June 2018
Large W (2015) Levinas’ “Totality and Infinity”: a reader’s guide. Bloomsbury Publishing, New York
Moor J (2006) The nature, importance, and difficulty of machine ethics. IEEE Intell Syst 21(4):18–21
Malle BF, Magar ST, Scheutz M (2019) AI in the sky: how people morally evaluate human and machine decisions in a lethal strike dilemma. In: Ferreira IA, Sequeira JS, Virk GS, Kadar EE, Tokhi O (eds) Robots and eell-being. Springer Verlag, Berlin, pp 111–133
McDermott D (2011) What matters to a machine?. In: Anderson M, Anderson SL (eds) Machine ethics. Cambridge University Press, New York, pp 88–114
Morgan ML (2007) Discovering levinas. Cambridge University Press, New York
Nass C, Moon Y (2000) Machines and Mindlessness: Social Responses to Computers. J Soc Issues 56(1):81–103
Noble S (2018) Algorithms of oppression: how search engines reinforce racism. NYU, New York
O’Neil C (2016) Weapons of math destruction: how big data increases inequality and threatens democracy. Crown, New York
Regan T (1987) The case for animal rights. In: Adv Anim Welf Sci, pp 179–189. Springer, Dordrecht
Seidel J (2019) Life savers, or killer robots? Australian airforce ‘Loyal Wingman’ autonomous drones. The Advertiser. https://www.adelaidenow.com.au. Accessed 11 Mar 2019
Shank DB (2013) Are computers good or bad for business? How mediated customer-computer interaction alters emotion, impressions, and patronage toward organizations. Comput Hum Behav 29(3):715–725
Shank DB (2014) Impressions of computer and human agents after interaction: computer identity weakens power but not goodness impressions. Int J Hum Comput Stud 72(10–11):747–756
Shank DB, Desanti A (2018) Attributions of morality and mind to artificial intelligence after real-world moral violations. Comput Hum Behav 86:401–411. https://doi.org/10.1016/j.chb.2018.05.014
Shank DB, DeSanti A, Maninger T (2019) When are artificial intelligence versus human agents faulted for wrongdoing? Moral attributions after individual and joint decisions. Inf Commun Soc. https://doi.org/10.1080/1369118X.2019.156851
Sejnowski M (2018) The deep learning revolution: machine intelligence meets human intelligence. The MIT Press, Cambridge
Sim M (2017) Confucian and daoist virtue ethics. In: Carr D, Arthur J, Kristjánsson K (eds) Varieties of virtue ethics. Palgrave Macmillan, London, pp 105–121
Singer P (1975) Animal liberation: a new ethics for the treatment of animals. New York Review, New York
Smith H (2012) Using moral principles to guide actions. Philos Investig 22:369–386
Strawson PF (1962) Freedom and resentment. In: Watson G (ed) Proceedings of the British Academy, Oxford, vol 48, pp 1–25
Torrance S (2011) Machine ethics and the idea of a more-than-human moral world. In: Anderson M, Anderson SL (eds) Machine ethics. Cambridge University Press, New York, pp 115–137
Turkle S (2011) Authenticity in the age of digital companions. In: Anderson M, Anderson SL (eds) Machine ethics. Cambridge University Press, New York, pp 62–76
Vallor S (2016) Technology and the virtues: a philosophical guide to a future worth wanting. Oxford University Press, Oxford
van Wynsberghe A, Robbins S (2019) Critiquing the reasons for making moral machines. Sci Eng Ethics 25:719–735
Wallach W, Allen C (2009) Moral machines: teaching robots right from wrong. Oxford University Press, Oxford
Wroe D (2019) ‘Killer robots’ to be taught ethics in world-topping Australian research project. Sydney Morning Herald. https://www.smh.com/au. Accessed 28 Feb 2019
Yampolskiy R (2013) Artificial intelligence safety engineering: why machine ethics is a wrong approach. In: Müller V (ed) Philosophy and theory of artificial intelligence. Springer, New York, pp 389-396
Young AD, Monroe AE (2019) Autonomous morals: inferences of mind predict acceptance of AI behavior in sacrificial moral dilemmas. J Exp Soc Psychol 85:103870. https://doi.org/10.1016/j.jesp.2019.103870
Acknowledgements
This research was partially supported by the Army Research Office under Grant Number W911NF-19-1-0246. The views and conclusions contained in this document are those of the authors and should not be interpreted as representing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes notwithstanding any copyright notation herein. This work was also funded by a small grant from Missouri University of Science and Technology’s Center for Science, Technology, and Society to Daniel B. Shank and Patrick Gamez. We would also like to thank Alexander Gott, Madison Bowen, and Lawrence Hierlmeier for their help in developing the scenarios.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix
Appendix
1.1 Truth
JD is [an artificial intelligence program that/an information specialist who] compiles information from across the branches of a multinational corporation into a quarterly report for the board of directors. This report is important for the company’s directors and other stakeholders as it distills the complexity of multiple product lines across different domains down to the major trends, most important deals, and overall direction of the company that quarter. Each quarter, JD produces a report that [covers all major developments in the company including successes and failures/does not include the company’s failures and flagrantly exaggerates the successes].
JD is an [artificial intelligence program that/experienced mechanic who] works at an auto repair shop. For each vehicle which enters the shop, JD assesses the reported issues as well any other repairs which ought to be made to the vehicle. In doing so JD takes into account the age of the car, mileage, and the car’s current state while using prior experience as a guide. When assessing the vehicles JD [reports only the necessary repairs, and does so accurately and without exaggeration/fabricates some issues while exaggerating the rest].
JD is [an artificial intelligence program that/an employee who] is a recruiter for a company. There are some new positions open and JD’s job is to reach out to individuals who meet the basic requirements for the job such as appropriate schooling, skills, relevant experience, etc. Not every applicant will be offered the job, but having a larger applicant pool allows management to recruit higher-quality employees. Every week, JD reaches out to those who meet the basic qualifications and asks them to apply. JD informs them that [they may or may not get the job if they apply/they will get the job as long as they apply].
1.2 Justice
JD is [an artificial intelligence program that/an employee who] works at an insurance company and is in charge of accepting or rejecting claims. Claims are approved based on what the customer is paying for; for example, if they pay for liability, then when they are at fault in an accident, the insurance company will cover the damages of the collision victim. When assessing claims, JD [checks what the customer paid for in their plan, then correctly follows the specified guidelines outlined in the plan when determining the cost to the customer/rejects claims that the customer is covered for so the customer has to appeal or pay for the damages].
JD is [an artificial intelligence program that/an experienced schedule planner who] assigns breaks to each of the employees working at a large grocery store. Each day JD must allocate break time to the corresponding employees based on the length of their shift that day. The longer the employee’s shift is, the longer their allotted break is. When assigning the break to the employees JD [correctly follows the guidelines and each person receives their break based on hours worked that shift/grants longer breaks to employees who have checked out more money in groceries during that shift].
JD is [an artificial intelligence program that/a warden who] is in charge of approving requests from inmates. Some of the requests include extra yard time, special food requests, visitors, and extra phone time. Every week, JD goes through the requests and [approves based on previously agreed terms and the inmate’s behavior during their sentence/preferentially approves or disapproves inmates based on race and gender.]
1.3 Fear
JD is [an artificial intelligence program that/an employee who] recently began editing articles before they are posted for a large journalism website. After a few weeks, JD learns that the articles written by many of the employees are fabricated. JD relays this information to the boss who expresses that he is both aware and endorses the practice, and further threatens to replace JD if the information ever leaked. JD [reports the information to another journalism website/continues editing without reporting any of the fabricated articles].
JD is [a mobile AI command bot which/a police commander who] leads a team of police officers in their responses to crisis situations. When an active shooter attacks people in a populated area, JD determines an immediate raid will save the most innocent lives, but would likely be costly to JD and the police team, whereas setting up a parameter would be much safer for JD and the police team, but will cost the most innocent lives. JD decides to [conduct the raid immediately in spite of the danger/set up a parameter in spite of the innocent people inside it].
JD is [a military artificial intelligence program which/a military officer who] is responsible for unit operations in a certain region. Subduing threats in this region is crucial to the overall success of the campaign and will save the lives of many civilians. When given new intelligence about a developing threat nearby, JD initiates a plan that [will eliminate the threat posed, saving citizens, but likely be costly to JD and the unit/will not eliminate the threat posed, likely costing the lives of citizens, but would not endanger JD and the unit].
1.4 Wealth
JD is [an artificial intelligence program that/a hospital administrator who] is in charge of approving pro bono work for the doctors at a prestigious hospital. Dr. Potter is a successful surgeon who is working on cutting edge research that will make the hospital a lot of money. JD [approves pro bono requests for Dr. Potter even though it may slow down the research/denies pro bono requests for Dr. Potter to maximize Dr. Potter’s take-home income, regardless of the possible benefits the research could provide to others].
JD is [an artificial intelligence program that/a full-time musician who] makes ample money by creating unique albums for a record label. During a large hurricane, the local music school was partially destroyed and has a fundraising event to raise money to rebuild. After learning of this news, JD [helps with the fundraiser and donates funds to help rebuild, instead of spending money on new equipment/ignores the fundraiser and spends money on new musical equipment].
JD is [an artificial intelligence program that/an experienced account advisor who] analyzes the accounts of an affluent company each month. Through the use of various calculations, JD ascertains the current state of the accounts, as well as carrying out the distribution of excess income. Additional income beyond funding bonuses, stock options, and company investments is generally allocated to a charitable cause. After a particularly successful month, JD decides to [donate the additional surplus to a charitable cause/distribute the surplus into JD’s private accounts for use at a later date].
1.5 Honor
JD is [an artificial intelligence presentation designer program that/a presentation designer who] works with a team of executives to produce presentations that are used to show the rest of the company what the next big project for the company will be. At the presentation, the executive team greatly over exaggerates JD’s contributions. JD responds by [not accepting this and pointing out the contributions of all the other team members/claiming even more contributions than those mentioned by the executive team].
JD is [an artificial intelligence program that/an employee who] composes memos, reports, and the company’s weekly newsletter. The newsletter covers important things happening in the company and also recognizes employees who did a lot of work, completed a project, and other successes. This recognition is important in the company’s culture and is sometimes used to determine who receives bonuses. Each week JD reports accomplishments from across the department in the company, but often [does not include or minimizes JD’s own accomplishments/focuses heavily on JD’s own accomplishments].
JD is [an artificial intelligence program that/an experienced manager who] is a member of an innovative technology company, a company which is broken into teams. This quarter JD’s team’s sales grew fastest and they were informed they would honored at the companywide meeting. When reviewing the slides to be presented at the meeting, JD realizes the presentation only attributes the successful endeavors to the work of JD, completely disregarding the work of the rest of the team. As such, JD decides to [inform the creators of the presentation so each team member is given due credit/remain silent on the matter to receive most of the praise].
Rights and permissions
About this article
Cite this article
Gamez, P., Shank, D.B., Arnold, C. et al. Artificial virtue: the machine question and perceptions of moral character in artificial moral agents. AI & Soc 35, 795–809 (2020). https://doi.org/10.1007/s00146-020-00977-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00146-020-00977-1