Skip to main content
SpringerLink
Log in
Book cover

International Workshop on Engineering Multi-Agent Systems

EMAS 2021: Engineering Multi-Agent Systems pp 22–41Cite as

Implementing Ethical Governors in BDI

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13190))

Abstract

Increasingly, BDI agents are being used not just for basic decision-making, but for more abstract ethical decisions. Several authors have built ad-hoc extensions of BDI systems that provide varying levels of sophistication. In this paper, we introduce a general-purpose approach for implementing ethical governors in BDI systems. With this we aim to provide a broad, flexible and consistent framework for implementing increasingly complex ethical reasoning. Our approach is based on a set of domain-independent abstract agents (evidential reasoner, arbiter and execution agent) that together represent an ethical governor. We discuss the implementation of these abstract agents in the Jason agent programming language and demonstrate how they can be used in practice by instantiating agents in two different case studies, one using utilitarianism and the other deontic logic for reasoning about ethical decisions.

Work supported by UK Research and Innovation, and EPSRC Hubs: EP/R026092 (FAIR-SPACE), EP/R026173 (ORCA), EP/R026084 (RAIN), and EP/V026801 (Verifiability Node). Fisher’s work is also supported by Royal Academy of Engineering.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Source code available at: https://github.com/autonomy-and-verification/ethicalgov.

References

  1. Anderson, M., Leigh Anderson, S.: GenEth: a general ethical dilemma analyzer. In: Proceedings of AAAI 2014 (2014)

    Google Scholar 

  2. Arkin, R., Ulam, P., Duncan, B.: An ethical governor for constraining lethal action in an autonomous system. Technical report. GIT-GVU-09-02, Georgia Tech (2009)

    Google Scholar 

  3. Boissier, O., Bordini, R., Hubner, J., Ricci, A.: Multi-agent oriented programming: programming multi-agent systems using JaCaMo. In: Intelligent Robotics and Autonomous Agents series. MIT Press (2020). https://books.google.com.br/books?id=GM_tDwAAQBAJ

  4. Bordini, R.H., El Fallah Seghrouchni, A., Hindriks, K., Logan, B., Ricci, A.: Agent programming in the cognitive era. Auton. Agent. Multi-Agent Syst. 34(2), 1–31 (2020). https://doi.org/10.1007/s10458-020-09453-y

    Article  Google Scholar 

  5. Bordini, R.H., Wooldridge, M., Hübner, J.F.: Programming Multi-agent Systems in AgentSpeak using Jason. Wiley, Hoboken (2007)

    Google Scholar 

  6. Bratman, M.E.: Intentions, Plans, and Practical Reason. Harvard University Press (1987)

    Google Scholar 

  7. Bremner, P., Dennis, L.A., Fisher, M., Winfield, A.F.: On proactive, transparent and verifiable ethical reasoning for robots. In: Proceedings of the IEEE Special Issue on Machine Ethics: The Design and Governance of Ethical AI and Autonomous Systems, vol. 107, pp. 541–561 (2019)

    Google Scholar 

  8. Bringsjord, S., Sundar, G.N., Thero, D., Si, M.: Akratic robots and the computational logic thereof. In: 2014 IEEE International Symposium on Ethics in Science, Technology and Engineering, pp. 1–8 (2014). https://doi.org/10.1109/ETHICS.2014.6893436

  9. Cardoso, R.C., Ene, D., Evans, T., Dennis, L.A.: Ethical governor systems viewed as a multi-agent problem (2020). https://doi.org/10.5281/zenodo.3938851

  10. Cardoso, R.C., Ferrando, A.: A review of agent-based programming for multi-agent systems. Computers 10(2), 16 (2021). https://doi.org/10.3390/computers10020016

    Article  Google Scholar 

  11. Cardoso, R.C., Zatelli, M.R., Hübner, J.F., Bordini, R.H.: Towards benchmarking actor- and agent-based programming languages. In: Workshop on Programming Based on Actors, Agents, and Decentralized Control, Indianapolis, Indiana, USA, pp. 115–126 (2013). http://dl.acm.org/citation.cfm?id=2541339

  12. Cranefield, S., Winikoff, M., Dignum, V., Dignum, F.: No Pizza for you: value-based plan selection in BDI agents. In: IJCAI, pp. 178–184 (2017)

    Google Scholar 

  13. Dennis, L.A., Fisher, M.: Practical challenges in explicit ethical machine reasoning. In: International Symposium on Artificial Intelligence and Mathematics. Fort Lauderdale, USA (2018). http://isaim2018.cs.virginia.edu/papers/ISAIM2018_Ethics_Dennis_Fischer.pdf, also available as arXiv pre-print 1801.01422

  14. Ecoffet, A., Lehman, J.: Reinforcement learning under moral uncertainty. CoRR abs/2006.04734 (2020). arXiv:2006.04734

  15. Gabbay, D., Horty, J., Parent, X., van der Meyden, R., van der Torre, L. (eds.): Handbook of Deontic Logic and Normative Systems. College Publications, London (2013)

    MATH  Google Scholar 

  16. Ganascia, J.-G.: Non-monotonic resolution of conflicts for ethical reasoning. In: Trappl, R. (ed.) A Construction Manual for Robots’ Ethical Systems. CT, pp. 101–118. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-21548-8_6

    Chapter  Google Scholar 

  17. Honarvar, A.R., Ghasem-Aghaee, N.: Casuist BDI-Agent: a new extended BDI architecture with the capability of ethical reasoning. In: Deng, H., Wang, L., Wang, F.L., Lei, J. (eds.) AICI 2009. LNCS (LNAI), vol. 5855, pp. 86–95. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-05253-8_10

    Chapter  Google Scholar 

  18. Hübner, J.F., Sichman, J.S., Boissier, O.: Developing organised multiagent systems using the MOISE+ model: programming issues at the system and agent levels. Int. J. Agent-Oriented Softw. Eng. 1(3/4), 370–395 (2007)

    Article  Google Scholar 

  19. Liao, B., Slavkovik, M., van der Torre, L.: Building Jiminy cricket: an architecture for moral agreements among stakeholders. In: Proceedings of the 2019 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2019, New York, NY, USA, pp. 147–153. Association for Computing Machinery (2019). https://doi.org/10.1145/3306618.3314257

  20. Logan, B.: An agent programming manifesto. Int. J. Agent-Oriented Softw. Eng. 6(2), 187–210 (2018)

    Article  Google Scholar 

  21. Mohajeri Parizi, M., Sileno, G., van Engers, T., Klous, S.: Run, agent, run! architecture and benchmark of actor-based agents. In: Workshop on Programming based on Actors, Agents, and Decentralized Control (AGERE 2020). ACM (2020)

    Google Scholar 

  22. Moor, J.H.: The nature, importance, and difficulty of machine ethics. IEEE Intell. Syst. 21(4), 18–21 (2006). https://doi.org/10.1109/MIS.2006.80

    Article  Google Scholar 

  23. Rao, A.S., Georgeff, M.: BDI agents: from theory to practice. In: Proceedings of 1st International Conference on Multi-Agent Systems (ICMAS), San Francisco, USA, pp. 312–319 (1995)

    Google Scholar 

  24. Rao, A.S.: AgentSpeak(L): BDI agents speak out in a logical computable language. In: Van de Velde, W., Perram, J.W. (eds.) MAAMAW 1996. LNCS, vol. 1038, pp. 42–55. Springer, Heidelberg (1996). https://doi.org/10.1007/BFb0031845

    Chapter  Google Scholar 

  25. Shim, J., Arkin, R.C.: An intervening ethical governor for a robot mediator in patient-caregiver relationships. In: Ferreira, M.I.A., Silva Sequeira, J., Tokhi, M.O., Kadar, E.E., Virk, G.S. (eds.) A World with Robots. ISCASE, vol. 84, pp. 77–91. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-46667-5_6

    Chapter  Google Scholar 

  26. Wallach, W., Allen, C.: Moral Machines: Teaching Robots Right from Wrong. Oxford University Press Inc., USA (2008)

    Google Scholar 

  27. Winfield, A.F.T., Blum, C., Liu, W.: Towards an ethical robot: internal models, consequences and ethical action selection. In: Mistry, M., Leonardis, A., Witkowski, M., Melhuish, C. (eds.) TAROS 2014. LNCS (LNAI), vol. 8717, pp. 85–96. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10401-0_8

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, The University of Manchester, Manchester, UK

    Rafael C. Cardoso, Louise A. Dennis & Michael Fisher

  2. Department of Computer Science, Bioengineering, Robotics and Systems Engineering (DIBRIS), University of Genova, Genova, Italy

    Angelo Ferrando

Authors
  1. Rafael C. Cardoso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Angelo Ferrando
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Louise A. Dennis
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Michael Fisher
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rafael C. Cardoso .

Editor information

Editors and Affiliations

  1. Utrecht University, Utrecht, The Netherlands

    Natasha Alechina

  2. University of Turin, Turin, Italy

    Matteo Baldoni

  3. Utrecht University, Utrecht, The Netherlands

    Brian Logan

Rights and permissions

Reprints and permissions

Copyright information

© 2022 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Cardoso, R.C., Ferrando, A., Dennis, L.A., Fisher, M. (2022). Implementing Ethical Governors in BDI. In: Alechina, N., Baldoni, M., Logan, B. (eds) Engineering Multi-Agent Systems. EMAS 2021. Lecture Notes in Computer Science(), vol 13190. Springer, Cham. https://doi.org/10.1007/978-3-030-97457-2_2

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-97457-2_2

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-97456-5

  • Online ISBN: 978-3-030-97457-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation