Skip to main content
SpringerLink
Log in

Machine Morality: From Harm-Avoidance to Human-Robot Cooperation

  • Conference paper
  • First Online:
Biomimetic and Biohybrid Systems (Living Machines 2020)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12413))

Included in the following conference series:

Abstract

We present a new computational framework for modeling moral decision-making in artificial agents based on the notion of ‘Machine Morality as Cooperation’. This framework integrates recent advances from cross-disciplinary moral decision-making literature into a single architecture. We build upon previous work outlining cognitive elements that an artificial agent would need for exhibiting latent morality, and we extend it by providing a computational realization of the cognitive architecture of such an agent. Our work has implications for cognitive and social robotics. Recent studies in human neuroimaging have pointed to three different decision-making processes, Pavlovian, model-free and model-based, that are defined by distinct neural substrates in the brain. Here, we describe how computational models of these three cognitive processes can be implemented in a single cognitive architecture by using the distributed and hierarchical organization proposed by the DAC theoretical framework. Moreover, we propose that a pro-social drive to cooperate exists at the Pavlovian level that can also bias the rest of the decision system, thus extending current state-of-the-art descriptive models based on harm-aversion.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Adolphs, R.: Cognitive neuroscience of human social behaviour. Nat. Rev. Neurosci. 4(3), 165–178 (2003)

    Article  Google Scholar 

  2. Arsiwalla, X.D., Freire, I.T., Vouloutsi, V., Verschure, P.: Latent morality in algorithms and machines. In: Martinez-Hernandez, U., Vouloutsi, V., Mura, A., Mangan, M., Asada, M., Prescott, T.J., Verschure, P.F.M.J. (eds.) Living Machines 2019. LNCS (LNAI), vol. 11556, pp. 309–315. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-24741-6_27

    Chapter  Google Scholar 

  3. Arsiwalla, X.D., Herreros, I., Moulin-Frier, C., Sanchez-Fibla, M., Verschure, P.: Is consciousness a control process? In: Artificial Intelligence Research and Development: Proceedings of the 19th International Conference of the Catalan Association for Artificial Intelligence, Barcelona, Catalonia, Spain, 19–21 October 2016, p. 233. IOS Press (2016)

    Google Scholar 

  4. Arsiwalla, X.D., Herreros, I., Moulin-Frier, C., Verschure, P.: Consciousness as an evolutionary game-theoretic strategy. In: Mangan, M., Cutkosky, M., Mura, A., Verschure, P.F.M.J., Prescott, T., Lepora, N. (eds.) Living Machines 2017. LNCS (LNAI), vol. 10384, pp. 509–514. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-63537-8_43

    Chapter  Google Scholar 

  5. Arsiwalla, X.D., Sole, R., Moulin-Frier, C., Herreros, I., Sanchez-Fibla, M., Verschure, P.F.: The morphospace of consciousness. arXiv preprint arXiv:1705.11190 (2017)

  6. Axenopoulos, A., et al.: A hybrid human-robot collaborative environment for recycling electrical and electronic equipment. In: 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pp. 1754–1759. IEEE (2019)

    Google Scholar 

  7. Balas, G.J., Doyle, J.C.: Robustness and performance trade-offs in control design for flexible structures. IEEE Trans. Control Syst. Technol. 2(4), 352–361 (1994)

    Article  Google Scholar 

  8. Botvinick, M., Ritter, S., Wang, J.X., Kurth-Nelson, Z., Blundell, C., Hassabis, D.: Reinforcement learning, fast and slow. Trends Cogn. Sci. 23(5), 408–422 (2019)

    Article  Google Scholar 

  9. Camerer, C.F.: Behavioral Game Theory: Experiments in Strategic Interaction. Princeton University Press, Princeton (2011)

    Book  Google Scholar 

  10. Crockett, M.J.: Models of morality. Trends Cogn. Sci. 17(8), 363–366 (2013). https://doi.org/10.1016/j.tics.2013.06.005

    Article  Google Scholar 

  11. Crockett, M.J., Siegel, J.Z., Kurth-Nelson, Z., Dayan, P., Dolan, R.J.: Moral transgressions corrupt neural representations of value. Nat. Neurosci. 20(6), 879–885 (2017). https://doi.org/10.1038/nn.4557

    Article  Google Scholar 

  12. Curry, O.S.: Morality as cooperation: a problem-centred approach. In: Shackelford, T.K., Hansen, R.D. (eds.) The Evolution of Morality. EP, pp. 27–51. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-19671-8_2

    Chapter  Google Scholar 

  13. Curry, O.S., Mullins, D.A., Whitehouse, H.: Is it good to cooperate. Curr. Anthropol. 60(1), 47–69 (2019)

    Article  Google Scholar 

  14. Dayan, P.: How to set the switches on this thing. Curr. Opin. Neurobiol. 22(6), 1068–1074 (2012)

    Article  Google Scholar 

  15. Floreano, D., Keller, L.: Evolution of adaptive behaviour in robots by means of Darwinian selection. PLoS Biol. 8(1), e1000292 (2010)

    Article  Google Scholar 

  16. Freire, I.T., Arsiwalla, X.D., Puigbò, J.Y., Verschure, P.: Modeling theory of mind in multi-agent games using adaptive feedback control. arXiv preprint arXiv:1905.13225 (2019)

  17. Freire, I.T., Moulin-Frier, C., Sanchez-Fibla, M., Arsiwalla, X.D., Verschure, P.F.: Modeling the formation of social conventions from embodied real-time interactions. PLoS ONE 15(6), e0234434 (2020)

    Article  Google Scholar 

  18. Freire, I.T., Puigbò, J.Y., Arsiwalla, X.D., Verschure, P.F.: Limits of multi-agent predictive models in the formation of social conventions. Artif. Intell. Res. Dev.: Curr. Chall. New Trends Appl. 308, 297 (2018)

    Google Scholar 

  19. Freire, I.T., Puigbò, J.-Y., Arsiwalla, X.D., Verschure, P.F.M.J.: Modeling the opponent’s action using control-based reinforcement learning. In: Vouloutsi, V., Halloy, J., Mura, A., Mangan, M., Lepora, N., Prescott, T.J., Verschure, P.F.M.J. (eds.) Living Machines 2018. LNCS (LNAI), vol. 10928, pp. 179–186. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-95972-6_19

    Chapter  Google Scholar 

  20. Greene, J.D.: The cognitive neuroscience of moral judgment. Cogn. Neurosci. 4, 1–48 (2009)

    Google Scholar 

  21. Greene, J.D., Nystrom, L.E., Engell, A.D., Darley, J.M., Cohen, J.D.: The neural bases of cognitive conflict and control in moral judgment. Neuron 44(2), 389–400 (2004)

    Article  Google Scholar 

  22. Greene, J.: Solving the trolley problem. A companion to experimental philosophy, pp. 175–178 (2016)

    Google Scholar 

  23. Huys, Q.J., Eshel, N., O’Nions, E., Sheridan, L., Dayan, P., Roiser, J.P.: Bonsai trees in your head: how the pavlovian system sculpts goal-directed choices by pruning decision trees. PLoS Comput. Biol. 8(3), e1002410 (2012)

    Article  MathSciNet  Google Scholar 

  24. Kant, I.: Groundwork of the metaphysic of morals. Trans. HJ Paton. 3rd edn. Harper Torchbooks, New York (1785)

    Google Scholar 

  25. Lieberman, M.D.: Social cognitive neuroscience: a review of core processes. Annu. Rev. Psychol. 58(1), 259–289 (2007). https://doi.org/10.1146/annurev.psych.58.110405.085654

    Article  Google Scholar 

  26. Maffei, G., Santos-Pata, D., Marcos, E., Sánchez-Fibla, M., Verschure, P.F.: An embodied biologically constrained model of foraging: from classical and operant conditioning to adaptive real-world behavior in DAC-X. Neural Netw. 72, 88–108 (2015). https://doi.org/10.1016/j.neunet.2015.10.004

    Article  Google Scholar 

  27. Mill, J.S.: Utilitarianism. Longmans, Green and Company (1895)

    Google Scholar 

  28. Moulin-Frier, C., Arsiwalla, X.D., Puigbò, J.Y., Sanchez-Fibla, M., Duff, A., Verschure, P.F.: Top-down and bottom-up interactions between low-level reactive control and symbolic rule learning in embodied agents. In: CoCo@ NIPS (2016)

    Google Scholar 

  29. Moulin-Frier, C., et al.: DAC-h3: a proactive robot cognitive architecture to acquire and express knowledge about the world and the self. IEEE Trans. Cogn. Dev. Syst. 10(4), 1005–1022 (2017)

    Article  Google Scholar 

  30. Perolat, J., Leibo, J.Z., Zambaldi, V., Beattie, C., Tuyls, K., Graepel, T.: A multi-agent reinforcement learning model of common-pool resource appropriation. In: Advances in Neural Information Processing Systems, pp. 3643–3652 (2017)

    Google Scholar 

  31. Peysakhovich, A., Lerer, A.: Consequentialist conditional cooperation in social dilemmas with imperfect information. arXiv preprint arXiv:1710.06975 (2017)

  32. Rand, D.G., Greene, J.D., Nowak, M.A.: Spontaneous giving and calculated greed. Nature 489(7416), 427–430 (2012)

    Article  Google Scholar 

  33. Rand, D.G., Nowak, M.A.: Human cooperation. Trends Cogn. Sci. 17(8), 413–425 (2013)

    Article  Google Scholar 

  34. Rand, D.G., et al.: Intuitive cooperation and the social heuristics hypothesis: evidence from 15 time constraint studies. In: Available at ssrn.com/abstract, vol. 2222683 (2013)

    Google Scholar 

  35. Sanchez-Fibla, M., et al.: Allostatic control for robot behavior regulation: a comparative rodent-robot study. Adv. Complex Syst. 13(03), 377–403 (2010)

    Article  MathSciNet  Google Scholar 

  36. Shum, M., Kleiman-Weiner, M., Littman, M.L., Tenenbaum, J.B.: Theory of minds: understanding behavior in groups through inverse planning. arXiv preprint arXiv:1901.06085 (2019)

  37. Tomasello, M., Vaish, A.: Origins of human cooperation and morality. Annu. Rev. Psychol. 64, 231–255 (2013)

    Article  Google Scholar 

  38. Verschure, P.F.: Synthetic consciousness: the distributed adaptive control perspective. Philosophical Trans. R. Soc. B: Biol. Sci. 371(1701), 20150448 (2016)

    Article  Google Scholar 

  39. Verschure, P.F., Pennartz, C.M., Pezzulo, G.: The why, what, where, when and how of goal-directed choice: neuronal and computational principles. Philos. Trans. R. Soc. B: Biol. Sci. 369(1655), 20130483 (2014)

    Article  Google Scholar 

  40. Verschure, P.F., Voegtlin, T., Douglas, R.J.: Environmentally mediated synergy between perception and behaviour in mobile robots. Nature 425(6958), 620–624 (2003)

    Article  Google Scholar 

  41. Vouloutsi, V., Lallée, S., Verschure, P.F.M.J.: Modulating behaviors using allostatic control. In: Lepora, N.F., Mura, A., Krapp, H.G., Verschure, P.F.M.J., Prescott, T.J. (eds.) Living Machines 2013. LNCS (LNAI), vol. 8064, pp. 287–298. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-39802-5_25

    Chapter  Google Scholar 

  42. Wallach, W., Allen, C.: Bottom-up and top-down approaches for modeling human moral faculties. In: Proceedings of the 2005 CogSci Workshop, pp. 149–159 (2005)

    Google Scholar 

  43. Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right From Wrong. Oxford University Press, Oxford (2008)

    Google Scholar 

  44. Wallach, W., Allen, C., Franklin, S.: Consciousness and ethics: artificially conscious moral agents. Int. J. Mach. Conscious. 3(01), 177–192 (2011)

    Article  Google Scholar 

  45. Wallach, W., Allen, C., Smit, I.: Machine morality: bottom-up and top-down approaches for modelling human moral faculties. AI Soc. 22(4), 565–582 (2008)

    Article  Google Scholar 

  46. Yoder, K.J., Decety, J.: The neuroscience of morality and social decision-making. Psychol. Crime Law 24(3), 279–295 (2018). https://doi.org/10.1080/1068316X.2017.1414817, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6372234/pdf/nihms-1500620.pdf

  47. Yu, H., Siegel, J.Z., Crockett, M.J.: Modeling morality in 3-D: decision-making, judgment, and inference. Top. Cogn. Sci. 11(2), 409–432 (2019). https://doi.org/10.1111/tops.12382

    Article  Google Scholar 

Download references

Acknowledgments

This research received funding from H2020-EU, ID: 820742.

Author information

Authors and Affiliations

  1. Institute for BioEngineering of Catalonia (IBEC), Barcelona, Spain

    Ismael T. Freire, Dina Urikh, Xerxes D. Arsiwalla & Paul F. M. J. Verschure

  2. Universitat Pompeu Fabra (UPF), Barcelona, Spain

    Ismael T. Freire, Dina Urikh & Xerxes D. Arsiwalla

  3. Barcelona Institue of Science and Technology (BIST), Barcelona, Spain

    Ismael T. Freire, Dina Urikh, Xerxes D. Arsiwalla & Paul F. M. J. Verschure

  4. Catalan Institute for Research and Advanced Studies (ICREA), Barcelona, Spain

    Paul F. M. J. Verschure

Authors
  1. Ismael T. Freire
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dina Urikh
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xerxes D. Arsiwalla
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paul F. M. J. Verschure
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ismael T. Freire .

Editor information

Editors and Affiliations

  1. SPECS, Institute for Bioengineering of Catalonia, Barcelona, Spain

    Vasiliki Vouloutsi

  2. SPECS, Institute for Bioengineering of Catalonia, Barcelona, Spain

    Anna Mura

  3. University of Freiburg, Freiburg, Germany

    Falk Tauber

  4. University of Freiburg, Freiburg, Germany

    Thomas Speck

  5. University of Sheffield, Sheffield, UK

    Tony J. Prescott

  6. SPECS, Institute for Bioengineering of Catalonia, Barcelona, Spain

    Paul F. M. J. Verschure

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Freire, I.T., Urikh, D., Arsiwalla, X.D., Verschure, P.F.M.J. (2020). Machine Morality: From Harm-Avoidance to Human-Robot Cooperation. In: Vouloutsi, V., Mura, A., Tauber, F., Speck, T., Prescott, T.J., Verschure, P.F.M.J. (eds) Biomimetic and Biohybrid Systems. Living Machines 2020. Lecture Notes in Computer Science(), vol 12413. Springer, Cham. https://doi.org/10.1007/978-3-030-64313-3_13

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-64313-3_13

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-64312-6

  • Online ISBN: 978-3-030-64313-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation