Abstract
In this paper, we propose a refinement of multiple-R [1], which is a reinforcement-learning based mechanism to create a social convention from a significantly large convention space for multi-agent systems. We focus on the language coordination problem, where agents develop a lexicon convention from scratch. As a lexicon is a set of mappings of concepts and words, the convention space is exponential to the number of concepts and words. We find that multiple-R suffers from local conventions, and refine it to the independent-R mechanism, which excludes neighbors’ rewards from the value update function, and thus avoids local conventions. We also explore how local conventions influence the dynamics of convention emergence. Extensive simulations verify that independent-R outperforms the state-of-the-art approaches, in the sense that a more widely adopted convention emerges in less time.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Wang, Y., Lu, W., Hao, J., Wei, J., Leung, H.-F.: Efficient convention emergence through decoupled reinforcement social learning with teacher-student mechanism. In: Proceedings of 17th International Conference on Autonomous Agents and MultiAgent Systems (2018)
Marchant, J.M., Griffiths, N., Leeke, M.: Manipulating conventions in a particle-based topology. In: Coordination, Organizations, Institutions and Norms in Agent Systems Workshop: A workshop of the 12th International Conference on Autonomous Agents and Multiagent Systems: AAMAS2015 (2015)
Sen, O., Sen, S.: Effects of social network topology and options on norm emergence. In: Padget, J., et al. (eds.) Coordination, Organizations, Institutions and Norms in Agent Systems V (2010)
Mihaylov, M., Tuyls, K., Nowé, A.: A decentralized approach for convention emergence in multi-agent systems. Auton. Agent. Multi-Agent Syst. 28, 479–779 (2014)
Airiau, S., Sen, S., Villatoro, D.: Emergence of conventions through sociallearning. Auton. Agent. Multi-Agent Syst. 28, 779–804 (2014)
Sen, S., Airiau, S.: Emergence of norms through social learning. In: Proceedings of 20th International Joint Conference on Artifical Intelligence (2007)
Hu, S., Fung Leung, H.: Achieving coordination in multi-agent systems by stable local conventions under community networks. In: Proceedings of 26th International Joint Conference on Artificial Intelligence (2017)
Hasan, M.R., Raja, A., Bazzan, A.: Fast convention formation in dynamic networks using topological knowledge. In: Proceedings of 29th AAAI Conference on Artificial Intelligence (2015)
Steels, L.: A self-organizing spatial vocabulary. Artif. Life 2, 319–332 (1995)
Hasselt, H.V.: Double q-learning. In: Advances in Neural Information Processing Systems 23. Curran Associates Inc. (2010)
Shoham, Y., Tennenholts, M.: Emergent conventions in multi-agent systems: initial experimental results and observations (preliminary report). In: Proceedings of Knowledge Representation and Reasoning (1992)
Franks, H., Griffiths, N., Jhumka, A.: Manipulating convention emergence using influencer agents. Auton. Agent. Multi-Agent Syst. 26(3), 315–353 (2013)
Salazar, N., Rodrigues Aguilar, J.A., Arcos, J.L.: Robust coordination in large convention spaces. AI Commun. 23(4), 357–372 (2010)
Watts, D.J., Strogatz, S.H.: Collective dynamics of ’small-world’ networks. Nature 393, 440–442 (1998)
Barabasi, A.-L., Albert, R.: Emergence of scaling in random networks. Science 286, 509–512 (1999)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, M., Leung, Hf., Hao, J. (2020). Faster Convention Emergence by Avoiding Local Conventions in Reinforcement Social Learning. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing. ICAISC 2020. Lecture Notes in Computer Science(), vol 12415. Springer, Cham. https://doi.org/10.1007/978-3-030-61401-0_64
Download citation
DOI: https://doi.org/10.1007/978-3-030-61401-0_64
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61400-3
Online ISBN: 978-3-030-61401-0
eBook Packages: Computer ScienceComputer Science (R0)