Skip to main content

Advertisement

SpringerLink
Log in

Autonomous Agents and Ethical Decision-Making

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Machine ethics, also known as artificial morality, is a newly emerging field concerned with ensuring appropriate behavior of machines toward humans and other machines. In this article, we discuss the importance of machine ethics and present a computational model of ethical decision-making for autonomous agents. The proposed model implements a mechanism for integrating the results of diverse assessments into a unique cue, and takes into account the agent’s preferences, good and bad past experiences, ethical rules, and current emotional state as the main factors involved in choosing the most appropriate option. The design of the model is based on theories and models developed in fields such as neuroscience, psychology, artificial intelligence, and cognitive informatics. In particular, the model attempts to emulate neural mechanisms of the human brain involved in ethical decision-making.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Fuster J. The prefrontal cortex. 4th ed. Waltham: Academic Press; 2008.

    Google Scholar 

  2. Kibele A. Non-consciously controlled decision making for fast motor reactions in sports: a priming approach for motor responses to non-consciously perceived movement features. Psychol Sport Exerc. 2006;7:591–610.

    Article  Google Scholar 

  3. Bechara A, Damasio H, Damasio A. Emotion, decision making and the orbitofrontal cortex. Cereb Cortex. 2000;10:295–307.

    Article  CAS  PubMed  Google Scholar 

  4. Wallis JD. Orbitofrontal cortex and Its contribution to decision-making. Annu Rev Neurosci. 2007;30:31–56.

    Article  CAS  PubMed  Google Scholar 

  5. Wallach W, Allen C, Smit I. Machine morality: bottom-up and top-down approaches for modelling human moral faculties. AI Soc. 2008;22:565–82.

    Article  Google Scholar 

  6. Wallach W. Implementing moral decision making faculties in computers and robots. AI Soc. 2008;22:463–75.

    Article  Google Scholar 

  7. Harman G. The nature of morality: an introduction to ethics. Oxford: Oxford University Press; 1977.

    Google Scholar 

  8. Wallach W, Franklin S, Allen C. A conceptual and computational model of moral decision making in human and artificial agents. Top Cogn Sci. 2010;2:454–85.

    Article  PubMed  Google Scholar 

  9. Fong T, Nourbakhsh I, Dautenhahn K. A survey of socially interactive robots. Robot Auton Syst. 2003;42:143–66.

    Article  Google Scholar 

  10. Czubenko M, Kowalczuk Z, Ordys A. Autonomous driver based on an intelligent system of decision-making. Cogn Comput 2015;7:569–581.

    Article  Google Scholar 

  11. Juha R, Jukka H, Vili K, Antti T, Matti P. Minotaurus: a system for affective human-robot interaction in smart environments. Cogn Comput. 2014;6:940–53.

    Article  Google Scholar 

  12. Paul W, Barbara LT. Affective robotics: modelling and testing cultural prototypes. Cogn Comput. 2014;6:814–40.

    Article  Google Scholar 

  13. Bechara A. The role of emotion in decision-making: evidence from neurological patients with orbitofrontal damage. Brain Cogn. 2004;55:30–40.

    Article  PubMed  Google Scholar 

  14. Damasio AR, Everitt BJ, Bishop D. The somatic marker hypothesis and the possible functions of the prefrontal cortex [and discussion]. Philos Trans R Soc Lond B Biol Sci. 1996;351:1413–20.

    Article  CAS  PubMed  Google Scholar 

  15. Loewenstein GF, Weber EU, Hsee CK, Welch N. Risk as feelings. Psychol Bull. 2001;127:267–310.

    Article  CAS  PubMed  Google Scholar 

  16. Tom SM, Fox CR, Trepel C, Poldrack RA. The neural basis of loss aversion in decision-making under risk. Science. 2007;315:515–8.

    Article  CAS  PubMed  Google Scholar 

  17. Tversky A, Kahneman D. Rational choice and the framing of decisions. J Bus. 1986;211:251–78.

    Article  Google Scholar 

  18. Gul FA. The joint and moderating role of personality and cognitive style on decision making. Account Rev. 1984;59:264–77.

    Google Scholar 

  19. Thatcher A, De La Cour A. Small group decision-making in face-to-face and computer-mediated environments: the role of personality. Behav Inf Technol. 2003;22:203–18.

    Article  Google Scholar 

  20. Broeders R, Van den Bos K, Müller PA, Ham J. Should I save or should I not kill? How people solve moral dilemmas depends on which rule is most accessible. J Exp Soc Psychol. 2011;47:923–34.

    Article  Google Scholar 

  21. Anderson M, Anderson SL. Machine ethics: creating an ethical intelligent agent. AI Mag. 2007;28:15–26.

    Google Scholar 

  22. Allen C, Wallach W, Smit I. Why machine ethics? Intell Syst. 2006;21:12–7.

    Article  Google Scholar 

  23. Wallach W. Robot minds and human ethics: the need for a comprehensive model of moral decision making. Ethics Inf Technol. 2010;12:243–50.

    Article  Google Scholar 

  24. Hagras H, Callaghan V, Colley M, Clarke G, Pounds-Cornish A, Duman H. Creating an ambient-intelligence environment using embedded agents. Intell Syst IEEE. 2004;19:12–20.

    Article  Google Scholar 

  25. Ostos R, Cervantes JF, Ramos FF, Castillo B, Occello M. Context-sensitive ecosystem of intelligent environments. In: Proceedings of the 8th international conference on intelligent environments (IE); 2012 June 26–29; Guanajuato, Mexico. IEEE Computer Society; 2012. pp. 72–79.

  26. Laird JE, Newell A, Rosenbloom PS. SOAR: an architecture for general intelligence. Artif Intell. 1987;33:1–64.

    Article  Google Scholar 

  27. Laird JE. The Soar cognitive architecture. Cambridge: The MIT Press; 2012.

    Google Scholar 

  28. Baars BJ, Franklin S. An architectural model of conscious and unconscious brain functions: global workspace theory and IDA. Neural Netw. 2007;20:955–61.

    Article  PubMed  Google Scholar 

  29. Best BJ, Lebiere C. Cognitive agents interacting in real and virtual worlds. In: Ron S, editor. Cognition and multi-agent interaction: from cognitive modeling to social simulation. Cambridge: Cambridge University Press; 2006. p. 186–218.

    Google Scholar 

  30. Allen C, Varner G, Zinser J. Prolegomena to any future artificial moral agent. J Exp Theor Artif Intell. 2000;12:251–61.

    Article  Google Scholar 

  31. Breazeal C. Toward sociable robots. Robot Auton Syst. 2003;42:167–75.

    Article  Google Scholar 

  32. Kaplan F. Talking AIBO: First experimentation of verbal interactions with an autonomous four-legged robot. Learning to behave: interacting agents CELE-TWENTE Workshop on Language Technology 2000. pp. 57-63.

  33. Tsagarakis NG, Metta G, Sandini G, Vernon D, Beira R, Becchi F, Righetti L, Santos-Victor J, Ijspeert AJ, Carrozza MC. iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research. Adv Robot. 2007;21:1151–75.

    Article  Google Scholar 

  34. Hirai K, Hirose M, Haikawa Y, Takenaka T. The development of Honda humanoid robot. IEEE international conference on robotics and automation 1998. pp. 1321-1326.

  35. Gigerenzer G. Moral satisficing: rethinking moral behavior as bounded rationality. Top Cogn Sci. 2010;2:528–54.

    Article  PubMed  Google Scholar 

  36. Honarvar AR, Ghasem-Aghaee N. An artificial neural network approach for creating an ethical artificial agent. In: Proceedings of the IEEE international symposium on computational intelligence in robotics and automation (CIRA); 2009 December 15–18; Daejeon, Korea. IEEE Computer Society; 2009. pp. 290–295.

  37. Dehghani M, Tomai E, Forbus KD, and Klenk M. An integrated reasoning approach to moral decision-making. In: Proceedings of the twenty-third AAAI conference on artificial intelligence; 2008 July 13–17; Chicago, USA. AAAI; 2008. pp. 1280–1286.

  38. Coelho H, da Rocha C, António C, Trigo P. Decision making for agent moral conducts. In: Proceedings of the INForum 2010 - Simpósio de Informática; 2010 September 9–10; Braga, Portugal. INForum; 2010. pp. 721732.

  39. Franklin S, Strain S, McCall R, Baars B. Conceptual commitments of the LIDA model of cognition. J Artif Gen Intell. 2013;4:1–22.

    Article  Google Scholar 

  40. Snaider J, McCall R, Franklin S. The LIDA framework as a general tool for AGI. In: Schmidhuber J, Thórisson KR, Moshe L, editors. Artifical general intelligence. Berlin: Springer; 2011. p. 133–42.

    Chapter  Google Scholar 

  41. Honarvar AR, Ghasem-Aghaee N. Casuist BDI-agent: a new extended BDI architecture with the capability of ethical reasoning. In: Deng H, Wang L, Wang FL, Lei J, editors. Artificial intelligence and computational intelligence. Berlin: Springer; 2009. p. 86–95.

    Chapter  Google Scholar 

  42. Borg JS, Hynes C, Van HJ, Grafton S, Sinnott-Armstrong W. Consequences, action, and intention as factors in moral judgments: an fMRI Investigation. J Cogn Neurosci. 2006;18:803–17.

    Article  Google Scholar 

  43. De Martino B, Kumaran D, Seymour B, Dolan RJ. Frames, biases, and rational decision-making in the human brain. Science. 2006;313:684–7.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Lu LC, Rose GM, Blodgett JG. The effects of cultural dimensions on ethical decision making in marketing: an exploratory study. J Bus Ethics. 1999;18:91–105.

    Article  Google Scholar 

  45. O’Fallon MJ, Butterfield KD. A review of the empirical ethical decision-making literature: 1996–2003. J Bus Ethics. 2005;59:375–413.

    Article  Google Scholar 

  46. Rolls ET. The orbitofrontal cortex and reward. Cereb Cortex. 2000;10:284–94.

    Article  CAS  PubMed  Google Scholar 

  47. O’Doherty J, Kringelbach ML, Rolls ET, Hornak J, Andrews C. Abstract reward and punishment representations in the human orbitofrontal cortex. 2001;4:95–102.

  48. Opris I, Bruce CJ. Neural circuitry of judgment and decision mechanisms. Brain Res Rev. 2005;48:509–26.

    Article  PubMed  Google Scholar 

  49. Ernst M, Paulus MP. Neurobiology of decision making: a selective review from a neurocognitive and clinical perspective. Biol Psychiatry. 2005;58:597–604.

    Article  PubMed  Google Scholar 

  50. Gold JI, Shadlen MN. The neural basis of decision making. Annu Rev Neurosci. 2007;30:535–74.

    Article  CAS  PubMed  Google Scholar 

  51. Schultz W, Tremblay L, Hollerman JR. Reward processing in primate orbitofrontal cortex and basal ganglia. Cereb Cortex. 2000;10:272–83.

    Article  CAS  PubMed  Google Scholar 

  52. Crescentini C, Seyed-Allaei S, Vallesi A, Shallice T. Two networks involved in producing and realizing plans. Neuropsychologia. 2012;50:1521–35.

    Article  PubMed  Google Scholar 

  53. Hoshi E, Tanji J. Area-selective neuronal activity in the dorsolateral prefrontal cortex for information retrieval and action planning. J Neurophysiol. 2004;91:2707–22.

    Article  PubMed  Google Scholar 

  54. Markic O. Rationality and emotions in decision making. Interdiscip Descr Complex Syst. 2009;7:54–64.

    Google Scholar 

  55. Greene JD, Nystrom LE, Engell AD, Darley JM, Cohen JD. The neural bases of cognitive conflict and control in moral judgment. Neuron. 2004;44:389–400.

    Article  CAS  PubMed  Google Scholar 

  56. Miller EK, Cohen JD. An integrative theory of prefrontal cortex function. Ann Rev Neurosci. 2001;24:167–202.

    Article  CAS  PubMed  Google Scholar 

  57. Chambers RA, Taylor JR, Potenza MN. Developmental neurocircuitry of motivation in adolescence: a critical period of addiction vulnerability. Am J Psychiatry. 2003;160:1041–52.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Volkow ND, Fowler JS. Addiction, a disease of compulsion and drive: involvement of the orbitofrontal cortex. Cerebral cortex. 2000;10:318–25.

    Article  CAS  PubMed  Google Scholar 

  59. Haber SN, Calzavara R. The cortico-basal ganglia integrative network: the role of the thalamus. Brain Res Bull. 2009;78:69–74.

    Article  PubMed  PubMed Central  Google Scholar 

  60. Kennerley SW, Walton ME, Behrens TEJ, Buckley MJ, Rushworth MFS. Optimal decision making and the anterior cingulate cortex. Nat Neurosci. 2006;9:940–7.

    Article  CAS  PubMed  Google Scholar 

  61. Van Staveren I. Beyond utilitarianism and deontology: ethics in economics. Rev Polit Econ. 2007;19:21–35.

    Article  Google Scholar 

  62. Gross CG, Rodman HR, Gochin PM, Colombo MW. Inferior temporal cortex as a pattern recognition device. In: Baum EB, editor. Computational Learning and Cognition: proceedings of the 3rd NEC research symposium. Philadelphia: Siam; 1992. p. 44–73.

  63. Rodman HR. Development of inferior temporal cortex in the monkey. Cereb Cortex. 1994;4:484–98.

    Article  CAS  PubMed  Google Scholar 

  64. Frey S, Kostopoulos P, Petrides M. Orbitofrontal involvement in the processing of unpleasant auditory information. Eur J Neurosci. 2000;12:3709–12.

    Article  CAS  PubMed  Google Scholar 

  65. O’Doherty JP. Reward representations and reward-related learning in the human brain: insights from neuroimaging. Curr Opin Neurobiol. 2004;14:769–76.

    Article  PubMed  Google Scholar 

  66. Rogers RD, Ramnani N, Mackay C, Wilson JL, Jezzard P, Carter CS, Smith SM. Distinct portions of anterior cingulate cortex and medial prefrontal cortex are activated by reward processing in separable phases of decision-making cognition. Biol Psychiatry. 2004;55:594–602.

    Article  PubMed  Google Scholar 

  67. Rizzolatti G, Fadiga L, Gallese V, Fogassi L. Premotor cortex and the recognition of motor actions. Cogn Brain Res. 1996;3:131–41.

    Article  CAS  Google Scholar 

  68. Hoshi E, Tanji J. Integration of target and body-part information in the premotor cortex when planning action. Nature. 2000;408:466–70.

    Article  CAS  PubMed  Google Scholar 

  69. Ford RC, Richardson WD. Ethical decision making: a review of the empirical literature. J Bus Ethics. 1994;13:205–21.

    Article  Google Scholar 

  70. Ferrell OC, Gresham LG. A contingency framework for understanding ethical decision making in marketing. J Market. 1985;49:87–96.

    Article  Google Scholar 

  71. Wang Y, Liu D, Ruhe G. Formal description of the cognitive process of decision making. In: Proceedings of the third IEEE international conference on cognitive informatics; 2004 August 16–17; British Columbia, Canada. IEEE Computer Society; 2004. pp. 124–130.

  72. Yingxu W. A novel decision grid theory for dynamic decision-making. In: Proceedings of the fourth IEEE conference on cognitive informatics; 2005 August 8–10; California, USA. IEEE Computer Society; 2005. pp. 308-314.

  73. Wang Y. Inference algebra (IA): a denotational mathematics for cognitive computing and machine reasoning (I). Int J Cogn Inf Nat Intell. 2011;5:61–82.

    Article  Google Scholar 

  74. Wang Y. Inference Algebra (IA): a denotational mathematics for cognitive computing and machine reasoning (II). Int J Cogn Inf Nat Intell. 2012;6:21–47.

    Article  CAS  Google Scholar 

  75. Morten K. The human orbitofrontal cortex: linking reward to hedonic experience. Nat Rev Neurosci. 2005;6:691–702.

    Article  Google Scholar 

  76. Wang Y. RTPA: a denotational mathematics for manipulating intelligent and computational behaviors. Int J Cogn Inf Nat Intell. 2008;2:44–62.

    Article  Google Scholar 

  77. Forgas JP. Mood and judgment: the affect infusion model (AIM). Psychol Bull. 1995;117:39–66.

    Article  CAS  PubMed  Google Scholar 

  78. Hockey RJ, John MA, Clough PJ, Bdzola L. Effects of negative mood states on risk in everyday decision making. Cogn Emot. 2000;14:823–55.

    Article  Google Scholar 

  79. Ekman P. Basic emotions. Handb Cogn Emot. 1999;98:45–60.

    Google Scholar 

  80. Ekman P. Are there basic emotions? Psychol Rev. 1992;99:550–3.

    Article  CAS  PubMed  Google Scholar 

  81. Hunt SD, Vitell S. A general theory of marketing ethics. J Macromarket. 1986;6:5–16.

    Article  Google Scholar 

  82. Harsanyi JC. A theory of prudential values and a rule utilitarian theory of morality. Soc Choice Welfare. 1995;12:319–33.

    Article  Google Scholar 

  83. Vitell SJ, Nwachukwu SL, Barnes JH. The effects of culture on ethical decision-making: an application of Hofstede’s typology. J Bus Ethics. 1993;12:753–60.

    Article  Google Scholar 

  84. Hochschild AR. Emotion work, feeling rules, and social structure. Am J Sociol. 1979;85:551–75.

    Article  Google Scholar 

  85. Rodríguez LF, Ramos F, Wang Y. Cognitive computational models of emotions and affective behaviors. Int J Softw Sci Comput Intell (IJSSCI). 2012;2:41–63.

    Article  Google Scholar 

  86. Wang Y. On the cognitive processes of human perception with emotions, motivations, and attitudes. J Cogn Inf Nat Intell. 2007;1:1–13.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Cinvestav Guadalajara, Guadalajara, Mexico

    José-Antonio Cervantes, Sonia López & Félix Ramos

  2. Department of Computer Science, Instituto Tecnológico de Sonora, Sonora, Mexico

    Luis-Felipe Rodríguez

  3. Department of Well-Being and Sustainable Development, North Centre of the University of Guadalajara, Guadalajara, Mexico

    Francisco Robles

Authors
  1. José-Antonio Cervantes
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luis-Felipe Rodríguez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sonia López
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Félix Ramos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Francisco Robles
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to José-Antonio Cervantes.

Appendices

Appendix 1: Technical Information About First Case Study

The fuzzy rules used in the primary evaluation of option one “stop the car” in order to give way to a pedestrian are:

figure a

The fuzzy rules used in the primary evaluation of option two “continue driving” are:

figure b

The experiences given to the agent related to option one “stop the car” in order to give way to a pedestrian are:

figure c

The experiences given to the agent related to option two “continue driving” are:

figure d

Appendix 2: Fuzzy Ethical Rules Used in Both Cases

The fuzzy rules used by the agent in both cases are:

figure e

The norms and rules of the agent:

figure f

Appendix 3: Technical Information About Second Case

The fuzzy rules used in the primary evaluation of both options are:

figure g

The experiences of the agent in the second case are the following:

figure h

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cervantes, JA., Rodríguez, LF., López, S. et al. Autonomous Agents and Ethical Decision-Making. Cogn Comput 8, 278–296 (2016). https://doi.org/10.1007/s12559-015-9362-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-015-9362-8

Keywords

Search

Navigation