Skip to main content

Abstract

To effectively develop cooperative multiagent systems, we introduce an architecture that facilitates the agents’ dynamic adoption of conventions. It expands an existing agent model’s action selection architecture with a component that uses Natural Language Processing techniques. This component embeds conventions into agent interaction strategies to improve the predictability of other agents’ actions if all agents adopt the same conventions in their strategies.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    Some of these works focus on policies rather than conventions. These two concepts are similar, although policies sometimes have a more probabilistic flavour [19]: there is the option that an agent probabilistically chooses an alternative to the action recommended by the policy when exploring the space.

  2. 2.

    https://hanabi.github.io.

  3. 3.

    https://www.nltk.org.

  4. 4.

    https://stanfordnlp.github.io/stanza/index.html.

  5. 5.

    https://nlp.stanford.edu/software/lex-parser.shtml.

  6. 6.

    https://github.com/slavpetrov/berkeleyparser.

  7. 7.

    A detailed literature review is provided by [4].

  8. 8.

    https://github.com/goose3/goose3.

  9. 9.

    Another tool with the best performance in [3] was Newspaper. Unlike GOOSE, it was primarily designed for newspaper texts and cannot extract structured text.

  10. 10.

    If no ontology exists, then we can generate one from the text using parsing and concepts/relation extraction rules, which consider semantic and syntactic features of the words [26].

References

  1. Ågotnes, T., Van Der Hoek, W., Rodríguez-Aguilar, J.A., Sierra, C., Wooldridge, M.J.: On the logic of normative systems. In: IJCAI, vol. 7, pp. 1175–1180 (2007)

    Google Scholar 

  2. Balke, T., da Costa Pereira, C., Dignum, F., Lorini, E., Rotolo, A., Vasconcelos, W., Villata, S.: Norms in MAS: Definitions and Related Concepts. In: Andrighetto, G., Governatori, G., Noriega, P., van der Torre, L.W.N. (eds.) Normative Multi-Agent Systems, Dagstuhl Follow-Ups, vol. 4, pp. 1–31. Schloss Dagstuhl-Leibniz-Zentrum fuer Informatik, Dagstuhl, Germany (2013). https://doi.org/10.4230/DFU.Vol4.12111.1. http://drops.dagstuhl.de/opus/volltexte/2013/3998

  3. Barbaresi, A., Lejeune, G.: Out-of-the-box and into the ditch? multilingual evaluation of generic text extraction tools. In: Language Resources and Evaluation Conference (LREC 2020), pp. 5–13 (2020)

    Google Scholar 

  4. Bard, N., Foerster, J.N., Chandar, S., Burch, N., Lanctot, M., Song, H.F., Parisotto, E., Dumoulin, V., Moitra, S., Hughes, E., Dunning, I., Mourad, S., Larochelle, H., Bellemare, M.G., Bowling, M.: The hanabi challenge: a new frontier for ai research. Artif. Intell. 280 (2020). https://doi.org/10.1016/j.artint.2019.103216

  5. Berners-Lee, T., Hendler, J., Lassila, O.: The semantic web: a new form of web content that is meaningful to computers will unleash a revolution of new possibilities. ScientificAmerican.com (2001)

    Google Scholar 

  6. Boella, G., Di Caro, L., Robaldo, L.: Semantic relation extraction from legislative text using generalized syntactic dependencies and support vector machines. In: Theory, Practice, and Applications of Rules on the Web: 7th International Symposium, RuleML 2013, pp. 218–225. Springer (2013)

    Google Scholar 

  7. Canaan, R., Gao, X., Togelius, J., Nealen, A., Menzel, S.: Generating and adapting to diverse ad-hoc partners in hanabi. IEEE Trans. Games (2022)

    Google Scholar 

  8. Consortium, W.W.W.: Owl 2 Web Ontology Language Document Overview, 2nd edn. https://www.w3.org/TR/owl2-overview/

  9. da Costa Pereira, C., Tettamanzi, A.G., Villata, S., Liao, B., Malerba, A., Rotolo, A., van Der Torre, L.: Handling norms in multi-agent system by means of formal argumentation. J. Appl. Logics-IfCoLoG J. Logics Appl. 4(9), 1–35 (2017)

    Google Scholar 

  10. Cuzzolin, F., Morelli, A., Cirstea, B., Sahakian, B.J.: Knowing me, knowing you: theory of mind in ai. Psychol. Med. 50(7), 1057–1061 (2020)

    Article  Google Scholar 

  11. Dragoni, M., Villata, S., Rizzi, W., Governatori, G.: Combining nlp approaches for rule extraction from legal documents. In: 1st Workshop on MIning and REasoning with Legal texts (MIREL 2016) (2016)

    Google Scholar 

  12. Ferraro, G., Lam, H.P.: Nlp techniques for normative mining. FLAP 8(4), 941–974 (2021)

    MathSciNet  Google Scholar 

  13. Ferraro, G., Lam, H.P., Tosatto, S.C., Olivieri, F., Islam, M.B., van Beest, N., Governatori, G.: Automatic extraction of legal norms: Eevaluation of natural language processing tools. In: Sakamoto, M., Okazaki, N., Mineshima, K., Satoh, K. (eds.) New Frontiers in Artificial Intelligence. pp. 64–81. Springer International Publishing (2020)

    Google Scholar 

  14. Foerster, J., Song, F., Hughes, E., Burch, N., Dunning, I., Whiteson, S., Botvinick, M., Bowling, M.: Bayesian action decoder for deep multi-agent reinforcement learning. In: Proceedings of the 36th International Conference on Machine Learning, vol. 97, pp. 1942–1951. PMLR (2019)

    Google Scholar 

  15. Gao, X., Singh, M.P.: Extracting normative relationships from business contracts. In: AAMAS, pp. 101–108. Citeseer (2014)

    Google Scholar 

  16. Gaur, S., Vo, N.H., Kashihara, K., Baral, C.: Translating simple legal text to formal representations. In: New Frontiers in Artificial Intelligence: JSAI-isAI 2014 Workshops, pp. 259–273. Springer (2015)

    Google Scholar 

  17. Governatori, G., Rotolo, A.: A conceptually rich model of business process compliance. In: Proceedings of the Seventh Asia-Pacific Conference on Conceptual Modelling, vol. 110, pp. 3–12. Citeseer (2010)

    Google Scholar 

  18. Gray, J., Lerer, A., Bakhtin, A., Brown, N.: Human-level performance in no-press diplomacy via equilibrium search (2020). arXiv:2010.02923

  19. Harsanyi, J.C.: Games with randomly disturbed payoffs: a new rationale for mixed-strategy equilibrium points. Internat. J. Game Theory 2(1), 1–23 (1973)

    Article  MathSciNet  Google Scholar 

  20. Hessel, M., Modayil, J., van Hasselt, H., Schaul, T., Ostrovski, G., Dabney, W., Horgan, D., Piot, B., Azar, M., Silver, D.: Rainbow: combining improvements in deep reinforcement learning. Proc. AAAI Conf. Artif. Intell. 32(1) (2018). https://doi.org/10.1609/aaai.v32i1.11796. https://ojs.aaai.org/index.php/AAAI/article/view/11796

  21. Hu, H., Lerer, A., Cui, B., Pineda, L., Brown, N., Foerster, J.: Off-belief learning. In: Proceedings of the 38th International Conference on Machine Learning, vol. 139, pp. 4369–4379 (2021)

    Google Scholar 

  22. Hu, H., Lerer, A., Peysakhovich, A., Foerster, J.: “other-play” for zero-shot coordination. In: Proceedings of the 37th International Conference on Machine Learning, vol. 119, pp. 4399–4410 (2020)

    Google Scholar 

  23. Hu, H., Wu, D.J., Lerer, A., Foerster, J., Brown, N.: Human-ai coordination via human-regularized search and learning (2022). arXiv:2210.05125

  24. Jacob, A.P., Wu, D.J., Farina, G., Lerer, A., Hu, H., Bakhtin, A., Andreas, J., Brown, N.: Modeling strong and human-like gameplay with kl-regularized search. In: Proceedings of the 39th International Conference on Machine Learning, vol. 162, pp. 9695–9728 (2022)

    Google Scholar 

  25. Kaptein, R., Serdyukov, P., De Vries, A., Kamps, J.: Entity ranking using wikipedia as a pivot. In: Proceedings of the 19th ACM International Conference on Information and Knowledge Management. p. 69–78. CIKM ’10, Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1871437.1871451

  26. Kumar, N., Kumar, M., Singh, M.: Automated ontology generation from a plain text using statistical and nlp techniques. Int. J. Syst. Assur. Eng. Manag. 7, 282–293 (2016)

    Article  Google Scholar 

  27. Levkovskyi, O., Li, W.: Generating predicate logic expressions from natural language. In: SoutheastCon 2021, pp. 1–8. IEEE (2021)

    Google Scholar 

  28. Longo, C.F., Longo, F., Santoro, C.: Caspar: Towards decision making helpers agents for iot, based on natural language and first order logic reasoning. Eng. Appl. Artif. Intell. 104, 104269 (2021)

    Article  Google Scholar 

  29. Lu, X., Liu, J., Gu, Z., Tong, H., Xie, C., Huang, J., Xiao, Y., Wang, W.: Parsing natural language into propositional and first-order logic with dual reinforcement learning. In: Proceedings of the 29th International Conference on Computational Linguistics, pp. 5419–5431 (2022)

    Google Scholar 

  30. Lupu, A., Cui, B., Hu, H., Foerster, J.: Trajectory diversity for zero-shot coordination. In: Proceedings of the 38th International Conference on Machine Learning, vol. 139, pp. 7204–7213 (2021)

    Google Scholar 

  31. McGuinness, D.L., Van Harmelen, F., et al.: Owl web ontology language overview. https://www.w3.org/TR/2004/REC-owl-features-20040210/

  32. Michel, M., Djurica, D., Mendling, J.: Identification of decision rules from legislative documents using machine learning and natural language processing. In: Proceedings of the 55th Hawaii International Conference on System Sciences, pp. 6247–6256 (2022)

    Google Scholar 

  33. Montes, N., Osman, N., Sierra, C.: Combining theory of mind and abduction for cooperation under imperfect information. In: Baumeister, D., Rothe, J. (eds.) Multi-Agent Systems, pp. 294–311. Springer International Publishing, Cham (2022)

    Chapter  Google Scholar 

  34. Montes, N., Osman, N., Sierra, C.: A computational model of ostrom’s institutional analysis and development framework. Artif. Intell. 311, 103756 (2022). https://doi.org/10.1016/j.artint.2022.103756

    Article  MathSciNet  Google Scholar 

  35. Morales, J., Lopez-Sanchez, M., Rodriguez-Aguilar, J.A., Vasconcelos, W., Wooldridge, M.: Online automated synthesis of compact normative systems. ACM Trans. Auton. Adapt. Syst. (TAAS) 10(1), 1–33 (2015)

    Article  Google Scholar 

  36. Olson, T., Forbus, K.D.: Learning norms via natural language teachings (2022). arXiv:abs/2201.10556. https://api.semanticscholar.org/CorpusID:244305883

  37. Pehcevski, J., Vercoustre, A.M., Thom, J.A.: Exploiting locality of wikipedia links in entity ranking. In: Macdonald, C., Ounis, I., Plachouras, V., Ruthven, I., White, R.W. (eds.) Advances in Information Retrieval, pp. 258–269. Springer, Berlin, Heidelberg (2008)

    Chapter  Google Scholar 

  38. Rao, A.S.: Agentspeak(l): Bdi agents speak out in a logical computable language. In: Van de Velde, W., Perram, J.W. (eds.) Agents Breaking Away, pp. 42–55. Springer, Berlin, Heidelberg (1996)

    Chapter  Google Scholar 

  39. Sanagavarapu, L.M., Iyer, V., Reddy, R.: A deep learning approach for ontology enrichment from unstructured text (2021). arXiv:2112.08554

  40. Shih, A., Sawhney, A., Kondic, J., Ermon, S., Sadigh, D.: On the critical role of conventions in adaptive human-ai collaboration (2021)

    Google Scholar 

  41. Sleimi, A., Sannier, N., Sabetzadeh, M., Briand, L., Dann, J.: Automated extraction of semantic legal metadata using natural language processing. In: 2018 IEEE 26th International Requirements Engineering Conference (RE), pp. 124–135 (2018). https://doi.org/10.1109/RE.2018.00022

  42. Treutlein, J., Dennis, M., Oesterheld, C., Foerster, J.: A new formalism, method and open issues for zero-shot coordination (2023)

    Google Scholar 

  43. Tucker, M., Zhou, Y., Shah, J.: Adversarially guided self-play for adopting social conventions (2020)

    Google Scholar 

  44. de Weerd, H., Verbrugge, R., Verheij, B.: Higher-order social cognition in the game of rock-paper-scissors: A simulation study. In: Bonanno, G., van Ditmarsch, H., van der Hoek, W. (eds.) Proceedings of the 10th Conference on Logic and the Foundations of Game and Decision Theory, pp. 218–232 (2012)

    Google Scholar 

  45. de Weerd, H., Verheij, B.: The advantage of higher-order theory of mind in the game of limited bidding. In: van Eijck, J., Verbrugge, R. (eds.) CEUR Workshop Proceedings, vol. 751, pp. 149–164 (2011)

    Google Scholar 

  46. Wyner, A., Peters, W.: On rule extraction from regulations. In: Legal knowledge and information systems, pp. 113–122. IOS Press (2011)

    Google Scholar 

Download references

Acknowledgements

This research is conducted under the REDI Program, a project that has received funding from the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement no. 101034328. This paper reflects only the author’s view and the Research Executive Agency is not responsible for any use that may be made of the information it contains. This research also receives support from the VALAWAI project (Horizon #101070930) and the VAE project (Grant no. TED2021-131295B-C31) funded by MCIN/AEI/10.13039/501100011033.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Shuxian Pan .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pan, S., Sierra, C. (2023). Towards Convention-Based Game Strategies. In: Fornara, N., Cheriyan, J., Mertzani, A. (eds) Coordination, Organizations, Institutions, Norms, and Ethics for Governance of Multi-Agent Systems XVI. COINE 2023. Lecture Notes in Computer Science(), vol 14002. Springer, Cham. https://doi.org/10.1007/978-3-031-49133-7_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-49133-7_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-49132-0

  • Online ISBN: 978-3-031-49133-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics