Skip to main content
SpringerLink
Log in
Book cover

European Conference on Multi-Agent Systems

EUMAS 2022: Multi-Agent Systems pp 383–402Cite as

Integrating Quantitative and Qualitative Reasoning for Value Alignment

  • Conference paper
  • First Online:

  • 465 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13442))

Abstract

Agents that focus only on achieving their own goals may cause significant harm to society. As a result, when deciding which actions to perform, agents have to consider societal values and how their actions impact these values—the ‘value alignment problem’. There is therefore a need to integrate quantitative machine reasoning with an ability to reason about qualitative human values. In this paper, we present a novel framework for value-based reasoning that aims to bridge the gap between these two modes of reasoning. In particular, our framework extends the theory of grading to model how societal values can trade off with each other or with the agent’s goals. Furthermore, our framework introduces the use of hyperreal numbers to represent both quantitative and qualitative aspects of reasoning and help address the value alignment problem.

This work was supported by UKRI (EP/S023356/1) and EPSRC (EP/R033188/1).

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    Note that this is a generalisation of the standard representation of value-based argumentation, where the total order over values, combined with the strong assumption that each action promotes exactly one value, yields a total order over the actions [2]. By requiring only a partial order over the values, we can relax these underlying strong assumptions.

  2. 2.

    As is characteristic of many non-consequentialist ethical theories [21].

  3. 3.

    This is because of the transfer principle. See Sect. 3.

  4. 4.

    However, second-order statements may not transfer to the hyperreals in the same way. For example, the second order statement that there is no number x such that \(1 + 1 + 1 ... < x\) doesn’t carry over to the hyperreals.

  5. 5.

    This can be used to model values the agent is apathetic towards (zero weight) or is even hostile to (negative weight).

  6. 6.

    Hyperreal numbers are not the only such number system; for example, surreal numbers [10] are also a non-Archimedean extension of real numbers. We have decided to use hyperreal numbers because of the transfer principle.

References

  1. Atkinson, K., Bench-Capon, T.J.M.: Taking account of the actions of others in value-based reasoning. Artif. Intell. 254, 1–20 (2018) https://doi.org/10.1016/j.artint.2017.09.002, https://doi.org/doi.org/10.1016/j.artint.2017.09.002

  2. Atkinson, K., Bench-Capon, T.J.M.: Value-based argumentation. FLAP. 8(6), 1543–1588 (2021). https://collegepublications.co.uk/ifcolog/?00048

  3. Balinski, M., Laraki, R.: A theory of measuring, electing, and ranking. Proc. Natl. Acad. Sci. 104(21), 8720–8725 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  4. Bench-Capon, T.J.M.: Value-based argumentation frameworks. In: Benferhat, S., Giunchiglia, E. (eds.) Proceedings of 9th International Workshop on Non-Monotonic Reasoning (NMR 2002), 19–21 April Toulouse, France, pp. 443–454 (2002)

    Google Scholar 

  5. Bench-Capon, T.J.M., Modgil, S.: Norms and value based reasoning: justifying compliance and violation. Artif. Intell. Law 25(1), 29–64 (2017). https://doi.org/10.1007/s10506-017-9194-9

  6. Bostrom, N.: Superintelligence: Paths, Dangers, 1st edn. Strategies. Oxford University Press Inc, USA (2014)

    Google Scholar 

  7. Bostrom, N., et al.: Infinite ethics. Anal. Metaphys. 10, 9–59 (2011)

    Google Scholar 

  8. Brown, D.S., Schneider, J., Dragan, A.D., Niekum, S.: Value alignment verification. In: Meila, M., Zhang, T. (eds.) Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18–24 July 2021, Virtual Event. Proceedings of Machine Learning Research, vol. 139, pp. 1105–1115. PMLR (2021). http://proceedings.mlr.press/v139/brown21a.html

  9. Chang, R.: Value pluralism. In: Wright, J. (ed.) International Encyclopedia of the Social and Behavioral Sciences (Second Edition), pp. 21–26. Elsevier (2015)

    Google Scholar 

  10. Conway, J.H.: On Numbers and Games. AK Peters/CRC Press, Boca Raton (2000)

    Google Scholar 

  11. Cranefield, S., Winikoff, M., Dignum, V., Dignum, F.: No pizza for you: Value-based plan selection in BDI agents. In: Sierra, C. (ed.) Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, Melbourne, Australia, 19–25 August 2017, pp. 178–184. ijcai.org (2017). https://doi.org/10.24963/ijcai.2017/26, https://doi.org/10.24963/ijcai.2017/26

  12. Eckersley, P.: Impossibility and uncertainty theorems in AI value alignment (or why your AGI should not have a utility function). In: Espinoza, H., hÉigeartaigh, S.Ó., Huang, X., Hernández-Orallo, J., Castillo-Effen, M. (eds.) Workshop on Artificial Intelligence Safety 2019 co-located with the Thirty-Third AAAI Conference on Artificial Intelligence 2019 (AAAI-19), Honolulu, Hawaii, 27 January 2019. CEUR Workshop Proceedings, vol. 2301. CEUR-WS.org (2019). http://ceur-ws.org/Vol-2301/paper_7.pdf

    Google Scholar 

  13. Geffner, H.: Model-free, model-based, and general intelligence. In: Lang, J. (ed.) Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13–19, 2018, Stockholm, Sweden. pp. 10–17. ijcai.org (2018). https://doi.org/10.24963/ijcai.2018/2, https://doi.org/10.24963/ijcai.2018/2

  14. Hadfield-Menell, D., Russell, S.J., Abbeel, P., Dragan, A.D.: Cooperative inverse reinforcement learning. In: Lee, D.D., Sugiyama, M., von Luxburg, U., Guyon, I., Garnett, R. (eds.) Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems 2016, December 5–10, 2016, Barcelona, Spain. pp. 3909–3917 (2016), https://proceedings.neurips.cc/paper/2016/hash/c3395dd46c34fa7fd8d729d8cf88b7a8-Abstract.html

  15. Irving, G., Christiano, P.F., Amodei, D.: AI safety via debate. CoRR abs/1805.00899 (2018). http://arxiv.org/abs/1805.00899

  16. Keisler, H.J.: The hyperreal line. In: Ehrlich, P. (eds.) Real Numbers, Generalizations of the Reals, and Theories of Continua. Synthese Library, vol. 242, pp. 207–237. Springer, Dordrecht (1994). https://doi.org/10.1007/978-94-015-8248-3_8

  17. Little, I.M.: Social choice and individual values. J. Polit. Econ. 60(5), 422–432 (1952)

    Article  Google Scholar 

  18. Mercuur, R., Dignum, V., Jonker, C.M.: The value of values and norms in social simulation. J. Artif. Soc. Soc. Simul. 22(1), 9 (2019). https://doi.org/10.18564/jasss.3929

  19. Morreau, M.: Grading in groups. Econ. Philos. 32(2), 323–352 (2016)

    Article  Google Scholar 

  20. Noothigattu, R., et al.: Teaching AI agents ethical values using reinforcement learning and policy orchestration. IBM J. Res. Dev. 63(4/5), 2:1–2:9 (2019). https://doi.org/10.1147/JRD.2019.2940428

  21. Peterson, M.: A royal road to consequentialism? Ethical Theory Moral Pract. 13(2), 153–169 (2010)

    Article  Google Scholar 

  22. Popova, A., Regenwetter, M., Mattei, N.: A behavioral perspective on social choice. Ann. Math. Artif. Intell. 68(1), 5–30 (2013)

    Article  MathSciNet  Google Scholar 

  23. Raz, J.: Practical Reasoning. Oxford University Press, Oxford (1978)

    Google Scholar 

  24. Robinson, A.: Non-standard Analysis. Princeton University Press, Princeton (2016)

    Google Scholar 

  25. Schwartz, S.H.: An overview of the Schwartz theory of basic values. Online Read. Psychol. Cult. 2(1), 1–20 (2012)

    Google Scholar 

  26. Serramia, M., López-Sánchez, M., Moretti, S., Rodríguez-Aguilar, J.A.: On the dominant set selection problem and its application to value alignment. Auton. Agents Multi Agent Syst. 35(2), 42 (2021). https://doi.org/10.1007/s10458-021-09519-5

  27. Serramia, M., et al.: Exploiting moral values to choose the right norms. In: Furman, J., Marchant, G.E., Price, H., Rossi, F. (eds.) Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society, AIES 2018, New Orleans, LA, USA, 02–03 February 2018, pp. 264–270. ACM (2018). https://doi.org/10.1145/3278721.3278735

  28. Szabo, J., Such, J.M., Criado, N.: Understanding the role of values and norms in practical reasoning. In: Bassiliades, N., Chalkiadakis, G., de Jonge, D. (eds.) EUMAS/AT -2020. LNCS (LNAI), vol. 12520, pp. 431–439. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-66412-1_27

    Chapter  Google Scholar 

  29. Von Neumann, J., Morgenstern, O.: Theory of Games and Economic Behavior. Princeton University Press, Princeton (2007)

    Google Scholar 

  30. Zurek, T.: Goals, values, and reasoning. Expert Syst. Appl. 71, 442–456 (2017) https://doi.org/10.1016/j.eswa.2016.11.008

  31. Zurek, T., Mokkas, M.: Value-based reasoning in autonomous agents. Int. J. Comput. Intell. Syst. 14(1), 896–921 (2021). https://doi.org/10.2991/ijcis.d.210203.001

Download references

Author information

Authors and Affiliations

  1. King’s College London, London, UK

    Jazon Szabo, Jose M. Such & Sanjay Modgil

  2. Universitat Politecnica de Valencia, Valencia, Spain

    Natalia Criado

Authors
  1. Jazon Szabo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jose M. Such
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Natalia Criado
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Sanjay Modgil
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jazon Szabo .

Editor information

Editors and Affiliations

  1. University of Düsseldorf, Düsseldorf, Germany

    Dorothea Baumeister

  2. University of Düsseldorf, Düsseldorf, Germany

    Jörg Rothe

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Szabo, J., Such, J.M., Criado, N., Modgil, S. (2022). Integrating Quantitative and Qualitative Reasoning for Value Alignment. In: Baumeister, D., Rothe, J. (eds) Multi-Agent Systems. EUMAS 2022. Lecture Notes in Computer Science(), vol 13442. Springer, Cham. https://doi.org/10.1007/978-3-031-20614-6_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-20614-6_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-20613-9

  • Online ISBN: 978-3-031-20614-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation