Skip to main content
SpringerLink
Log in

Implementing moral decision making faculties in computers and robots

  • Original Article
  • Published:
AI & SOCIETY Aims and scope Submit manuscript

Abstract

The challenge of designing computer systems and robots with the ability to make moral judgments is stepping out of science fiction and moving into the laboratory. Engineers and scholars, anticipating practical necessities, are writing articles, participating in conference workshops, and initiating a few experiments directed at substantiating rudimentary moral reasoning in hardware and software. The subject has been designated by several names, including machine ethics, machine morality, artificial morality, or computational morality. Most references to the challenge elucidate one facet or another of what is a very rich topic. This paper will offer a brief overview of the many dimensions of this new field of inquiry.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adams B, Cynthia Breazeal, Rodney A, Brooks, Brian Scassellati (2000) Humanoid robots: a new kind of tool. IEEE Intell Syst Appl. Special issue on humanoid robotics 15(4):25–31

  • Allen C (2002) Calculated morality: ethical computing in the limit. In: Smit I, Lasker G (eds) Cognitive, emotive and ethical aspects of decision making and human action, vol I. IIAS, Windsor

  • Allen C, Varner G, Zinser J (2000) Prolegomena to any future artificial moral agent. J Exp Theor Artif Intell 12:251–261

    Article  MATH  Google Scholar 

  • Allen C, Smit I, Wallach W (2006) Artificial morality: top–down, bottom–up and hybrid approaches. Ethics N Inf Technol 7:149–155

    Article  Google Scholar 

  • Anderson M, Anderson S, Armen C (2006) An approach to computing ethics. IEEE Intell Syst 21(4):56–63

    Article  Google Scholar 

  • Asimov I (1950) I robot. Gnome Press, New York

    Google Scholar 

  • Asimov I (1985) Robots and empire. Grafton Books, London

    Google Scholar 

  • Beauchamp and Childress (2001) Principles of biomedical ethics, 5th edn. Oxford University Press, New York

  • Boella G, van der Torre L, Verhagen H (2005) Introduction to normative multiagent systems. In: Proceedings of the symposium on normative multi-agent systems, AISB-05. University of Hertfordshire, Hatfield, pp 1–7

  • Breazeal C (2002) Designing sociable robots. MIT, Cambridge

    Google Scholar 

  • Brooks R (2002) Flesh and machines. Pantheon Press, New York

    Google Scholar 

  • Calverley D (2005) Additional thoughts concerning the legal status of a non-biological machine. In: Anderson M, Anderson S, Armen C (eds) Machine ethics. AAAI Press, Menlo Park. Technical Report FS-05-06

  • Clarke R (1993, 1994) Asimov’s laws of robotics: implications for information technology. Published in two parts. IEEE Comput 26 (12) 53–61 and 27 (1) 57–66

    Google Scholar 

  • de Garis H (2005) The Artilect war: cosmists vs terrans: a bitter controversy concerning whether humans should build godlike massively intelligent machines. ETC Publications, Palm Springs

  • Damasio A (1995) Descartes’ error. Picador, London

    Google Scholar 

  • Danielson P (1992) Artificial morality: virtuous robots for virtual games. Routledge, New York

    Google Scholar 

  • Danielson P (1998) Modeling rationality, morality and evolution. Oxford University Press, Oxford

    Google Scholar 

  • DeSousa R (1987) The rationality of emotions. MIT Press, Cambridge

    Google Scholar 

  • Floridi L, Sanders JW (2004) On the morality of artificial agents. Minds Mach 14(3):349–379

    Article  Google Scholar 

  • Franklin S (2003) IDA: a conscious artifact? In Holland O (ed) J Conscious Stud. Special issue on machine consciousness 10(4–5):47–66

  • Friedman B, Kahn P (1992) Human agency and responsible computing: implications for computer system design. J Syst Softw 17(1):7–14

    Article  Google Scholar 

  • Gips J (1991) Towards the ethical robot. In: Ford K, Glymour C, Hayes P (eds) Android epistemology. MIT, Cambridge, pp 243–252

  • Goleman D (1995) Emotional intelligence. Bantam Books, New York

    Google Scholar 

  • Guarini M (2006) Particularism and classification and reclassification of moral cases. IEEE Intell Syst 21(4):22–28

    Article  Google Scholar 

  • Holland O (ed) (2003) J Conscious Stud. Special issue on Machine Consciousness. 10(4–5)

  • Joy W (2000) Why the future does not need us. Wired Magazine 8(4)

  • Kennedy C (2000) Reducing indifference: steps towards autonomous agents with human concerns. Artificial intelligence, ethics and (quasi-) human rights. In: Proceedings of AISB-2000 Workshop. University of Birmingham, Birmingham pp 7–16

  • Kennedy C (2004) Agents for trustworthy ethical assistance. In: Smit I, Lasker G and Wallach W (eds), Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence, vol III. IIAS, Windsor

  • Knaus W (2002) APACHE 1978–2001: the development of a quality assurance system based on prognosis. Arch Surg 137(1):37–41

    Article  Google Scholar 

  • Kraus S (2001) Strategic negotiations in multiagent environments. MIT, Cambridge

    Google Scholar 

  • Kurzweil R (1999) The Age of spiritual machines: when computers exceed human intelligence. Viking, New York

    Google Scholar 

  • Kurzweil R (2002) in Richards JW (ed) Are we spiritual machines? Ray Kurzweil vs. the critics of strong A.I. Discovery Institute Press

  • Kurzweil R (2005) The Singularity is near. Viking, New York

    Google Scholar 

  • Lang C (2002) Ethics for artificial intelligence. http://www.philosophy.wisc.edu/lang/AIEthics/index.htm

  • Moravec H (1998) Robot: mere machine to transcendent mind. Oxford University Press, Oxford

    Google Scholar 

  • Nissenbaum H (1996) Accountability in a computerized society. Sci Eng Ethics 2:25–42

    Article  Google Scholar 

  • Nolfi N (1998) Evolutionary robotics: exploiting the full power of self-organization. Connection Science (10) 3–4. 167–183

    Google Scholar 

  • Picard R (1997) Affective computing. MIT, Cambridge

    Google Scholar 

  • Pollack J (2004) Seven questions for the age of robots”, talk given at Yale University 4 February 2004. http://www.jordanpollack.com/sevenlaws.htm

  • Scassellati B (2001) Foundations for a theory of mind for a humanoid robot. PhD Thesis. Department of Electrical Engineering and Computer Science, MIT. http://www.ai.mit.edu/projects/lbr/hrg/2001/scassellati-phd.pdf

  • Simon HA (1982) Models of bounded rationality. MIT, Cambridge

    Google Scholar 

  • Smit I (2003) Robots, Quo Vadis. In: Smit I, Lasker G, Wallach W (eds) Cognitive, emotive and ethical aspects of decision making in humans and in artificial intellignce. IIAS, Windsor

  • Salovey P, Mayer JD (1990) Emotional intelligence. Imagin Cogn Pers 9:185–211

  • Turing A (1950) Computing machinery and intelligence. Mind 49:433–460

    Article  MathSciNet  Google Scholar 

  • Wallach W (2003) Robot morals and human ethics. In: Smit I, Lasker L, Wallach W (eds) Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence, vol II. IIAS, Windsor

  • Wallach W (2004) Artificial morality: bounded rationality, bounded morality and emotions. In: Smit I, Lasker G, Wallach W (eds) Cognitive, emotive and ethical aspects of decision making in humans and in artificial intelligence, vol III. IIAS, Windsor

  • Weckert J (2005) Trusting agents. In: Ethics of new information technology: CEPE2005. Enschede, Holland pp 407–412

  • Whitby B, Oliver K (2000) How to avoid a robot takeover: political and ethical choices in the design and introduction of intelligent artifacts. Artificial intelligence, ethics and (quasi-) human rights. In: Proceedings of AISB-2000 Workshop. University of Birmingham, Birmingham pp 53–58

Download references

Acknowledgments

I wish to thank Colin Allen and Iva Smit for our shared research in developing a framework for the study of machine morality. Conversations and suggestions from Steve Torrance, David Calverley, Karl MacDormand, Brian Scasselatti, Michael and Susan Anderson, and Rosalind Picard have contributed to my understanding of some of the themes in this paper.

Author information

Authors and Affiliations

  1. Yale University Interdisciplinary Center for Bioethics, New Haven, CT, USA

    Wendell Wallach

Authors
  1. Wendell Wallach
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wendell Wallach.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Wallach, W. Implementing moral decision making faculties in computers and robots. AI & Soc 22, 463–475 (2008). https://doi.org/10.1007/s00146-007-0093-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00146-007-0093-6

Keywords

Search

Navigation