Skip to main content
SpringerLink
Log in

Emergence of conventions through social learning

Heterogeneous learners in complex networks

  • Published:
Autonomous Agents and Multi-Agent Systems Aims and scope Submit manuscript

Abstract

Societal norms or conventions help identify one of many appropriate behaviors during an interaction between agents. The offline study of norms is an active research area where one can reason about normative systems and include research on designing and enforcing appropriate norms at specification time. In our work, we consider the problem of the emergence of conventions in a society through distributed adaptation by agents from their online experiences at run time. The agents are connected to each other within a fixed network topology and interact over time only with their neighbours in the network. Agents recognize a social situation involving two agents that must choose one available action from multiple ones. No default behavior is specified. We study the emergence of system-wide conventions via the process of social learning where an agent learns to choose one of several available behaviors by interacting repeatedly with randomly chosen neighbors without considering the identity of the interacting agent in any particular interaction. While multiagent learning literature has primarily focused on developing learning mechanisms that produce desired behavior when two agents repeatedly interact with each other, relatively little work exists in understanding and characterizing the dynamics and emergence of conventions through social learning. We experimentally show that social learning always produces conventions for random, fully connected and ring networks and study the effect of population size, number of behavior options, different learning algorithms for behavior adoption, and influence of fixed agents on the speed of convention emergence. We also observe and explain the formation of stable, distinct subconventions and hence the lack of emergence of a global convention when agents are connected in a scale-free network.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Notes

  1. In game theory, a focal point is an equilibrium more likely to be chosen by the players because it seems special, natural or relevant to them, although other equilibria are equally good.

  2. It might seem that “rules of the road” are always fixed by authority, but historical records show that “Society often converges on a convention first by an informal process of accretion; later it is codified into law” [46].

  3. These results were published in [32].

  4. If the norm (\(G\),\(Y_L\)) has emerged and all agents play \(\epsilon \)-greedy with \(\epsilon =0.04\), we will observe the outcome (\(G\),\(Y_L\)) with a probability of \(\left( .96+\frac{.04}{2}\right) ^2\), (\(G\),\(G\)) and (\(Y_R\),\(Y_L\)) with a probability of \((.96+\frac{.04}{2})\cdot \frac{.02}{2}\) and (\(Y_R, G\)) with a probability of \(\left( \frac{.02}{2}\right) ^2\). Overall, we have a probability of 0.4804 to observe each (\(G\),\(Y_L\)) and (\(Y_R\),\(G\)) and a probability of 0.0196 to observe (\(G\),\(G\)) and (\(Y_R\),\(Y_L\)).

  5. The diameter of a graph is the largest number of vertices which must be traversed in order to travel from one vertex to another.

References

  1. Airiau, S., Saha, S., & Sen, S. (2007). Evolutionary tournament-based comparison of learning and non-learning algorithms for iterated games. Journal of Artificial Societies and Social Simulation, 10(3), 1–7.

    Google Scholar 

  2. Axelrod, R. (1986). An evolutionary approach to norms. The American Political Science Review, 80(4), 1095–1111.

    Article  Google Scholar 

  3. Axelrod, R. (1997). The complexity of cooperation: Agent-based models of conflict and cooperation. Princeton, NJ: Princeton University Press.

    Google Scholar 

  4. Banerjee, D., & Sen, S. (2007). Reaching pareto-optimality in prisoner’s dilemma using conditional joint action learning. Autonomous Agents and Multi-Agent Systems, 15, 91–108.

    Article  Google Scholar 

  5. Barabási, A., & Albert, R. (1999). Emergence of scaling in random networks. Science, 286, 509–512.

    Article  MathSciNet  Google Scholar 

  6. Boella, G., & Lesmo, L. (2002). A game theoretic approach to norms and agents. Cognitive Science Quarterly, 2(2–3), 492–512.

    Google Scholar 

  7. Boella, G., & van der Torre, L. (2003). Norm governed multiagent systems: The delegation of control to autonomous agents. In IAT ’03: Proceedings of the IEEE/WIC International Conference on Intelligent Agent Technology (p. 329). Washington, DC, USA: IEEE Computer Society.

  8. Bowling, M. (2005). Convergence and no-regret in multiagent learning. Advances in Neural Information Processing Systems, 17, 209–216.

    Google Scholar 

  9. Bowling, M. H., & Veloso, M. M. (2002). Multiagent learning using a variable learning rate. Artificial Intelligence, 136(2), 215–250.

    Article  MATH  MathSciNet  Google Scholar 

  10. Coleman, J. S. (1998). Foundations of social theory. Cambridge, MA: Harvard University Press.

    Google Scholar 

  11. Crandall, J. W., & Goodrich, M. A. (2005). Learning to compete, compromise, and cooperate in repeated general-sum games. In Proceedings of the 22nd International Conference on, Machine Learning (pp. 161–168).

  12. Delgado, J. (2002). Emergence of social conventions in complex networks. Artificial Intelligence, 141(1–2), 171–185.

    Article  MathSciNet  Google Scholar 

  13. Dignum, F., Kinny, D., & Sonenberg, L. (2002). From desires, obligations and norms to goals. Cognitive Science Quarterly, 2(3–4), 407–430.

    Google Scholar 

  14. Epstein, J. M. (2001). Learning to be thoughtless: Social norms and individual computation. Computational Economics, 18(1), 9–24.

    Article  MATH  Google Scholar 

  15. Fudenberg, D., & Levine, D. K. (1998). The theory of learning in games. Cambridge, MA: MIT Press.

    MATH  Google Scholar 

  16. García-Camino, A., Rodríguez-Aguilar, J. A., Sierra, C., & Vasconcelos, W. (2006). A rule-based approach to norm-oriented programming of electronic institutions. SIGecom Exchanges, 5(5), 33–40.

    Article  Google Scholar 

  17. Genesereth, M. R., Ginsberg, M. L., & Rosenschein, J. S. (1988). Cooperation without communication. Distributed Artificial Intelligence, 220–226.

  18. Hales, D., & Arteconi, S. (2006). Slacer: A self-organizing protocol for coordination in p2p networks. IEEE Intelligent Systems, 21(29), 35.

    Google Scholar 

  19. Kandori, M., & Rob, R. (1995). Evolution of equilibria in the long run: A general theory and applications. Journal of Economic Theory, 65(2), 383–414.

    Article  MATH  MathSciNet  Google Scholar 

  20. Kittock, J. E. (1995). Emergent conventions and the structure of multi-agent systems. In Proceedings of the 1993 Santa Fe Institute Complex Systems Summer School. Santa Fe Institute Studies in the Sciences of Complexity Lecture. Santa Fe Institute.

  21. Lakkaraju, K., & Gasser, L. (2006). Population and agent based models for language convergence. In Proceedings of the 21st National Conference on Artificial Intelligence, AAAI’06 (Vol. 2, pp. 1887–1888). Menlo Park: AAAI Press.

  22. Lewis, D. K. (1969). Convention: A philosophical study. Cambridge, MA: Harvard University Press.

    Google Scholar 

  23. Montanari, A., & Saberi, A. (2010). The spread of innovations in social networks. Proceedings of the National Academy of Sciences, 107(47), 20196–20201.

    Article  Google Scholar 

  24. Mukherjee, P., Airiau, S., & Sen, S. (2008). Norm emergence under constrained interactions in diverse societies. In Proceedings of the Seventh International Conference on Autonomous Agents and Multiagent Systems (AAMAS-08) (pp. 779–786). Estoril, Portugal.

  25. Myerson, R. B. (1991). Game theory: Analysis of conflict. Cambridge, MA: Harvard University Press.

    MATH  Google Scholar 

  26. Nowak, M. A., & May, R. M. (1992). Evolutionary games and spatial chaos. Nature, 359, 826–829.

    Article  Google Scholar 

  27. Nudelman, E., Wortman, J., Shoham, Y., & Leyton-Brown, K. (2004). Run the gamut: A comprehensive approach to evaluating game-theoretic algorithms. In S. A (Ed.), Proceedings of the Third International Joint Conference on Autonomous Agents and Multiagent Systems (AAMAS ’04) (pp. 880–887). Washington, DC, USA: IEEE Computer Society.

  28. Panait, L., & Luke, S. (2005). Cooperative multi-agent learning: The state of the art. Autonomous Agents and Multi-Agent Systems, 11(3), 387–434.

    Article  Google Scholar 

  29. Powers, R., Shoham, Y., & Vu, T. (2007). A general criterion and an algorithmic framework for learning in multi-agent systems. Machine Learning, 67(1–2), 45–76.

    Article  Google Scholar 

  30. Schelling, T. C. (1960). The strategy of conflict. Cambridge, MA: Harvard University Press.

    Google Scholar 

  31. Schweitzer, F., Zimmermann, J., & Mühlenbein, H. (2002). Coordination of decisions in a spatial agent model. Physica A, 303(1–2), 189–216.

    Article  MATH  Google Scholar 

  32. Sen, S., & Airiau, S. (2007). Emergence of norms through social learning. In Proceedings of the Twentieth International Joint Conference on Artificial Intelligence (IJCAI’07) (pp. 1057–1512).

  33. Shoham, Y., & Tennenholtz, M. (1997). On the emergence of social conventions: Modeling, analysis, and simulations. Artificial Intelligence, 94(1–2), 139–166.

    Article  MATH  Google Scholar 

  34. Tinnemeier, N., Dastani, M., & Meyer, J.-J. (2010). Programming norm change. In Proceedings of the 9th International Conference on Autonomous Agents and Multiagent Systems (AAMAS ’10) (Vol. 1, pp. 957–964). Richland, SC: International Foundation for Autonomous Agents and Multiagent Systems.

  35. Tuyls, K., & Nowé, A. (2005). Evolutionary game theory and multi-agent reinforcement learning. Knowledge Engineering Review, 20(1), 63–90.

    Article  Google Scholar 

  36. Urbano, P., Balsa, J., Antunes, L., & Moniz, L. (2009). Force versus majority: A comparison in convention emergence efficiency. In Coordination, Organizations, Institutions and Norms in Agent Systems IV: COIN 2008 International Workshops, COIN@AAMAS 2008, Estoril, Portugal, May 12, 2008 (pp. 48–63). Berlin: Springer.

  37. Vázquez-Salceda, J., Aldewereld, H., & Dignum, F. P. M. (2005). Norms in multiagent systems: From theory to practice. International Journal of Computer Systems Science & Engineering, 20(4), 225– 236.

    Google Scholar 

  38. Verhagen, H. (2001). Simulation of the learning of norms. Social Science Computer Review, 19(3), 296–306.

    Article  Google Scholar 

  39. Villatoro, D., Sen, S., & Sabater-Mir, J. (2009). Topology and memory effect on convention emergence. In S. A (Ed.), Proceedings of the International Conference of Intelligent Agent Technology (IAT). Hoboken, NJ: IEEE Press.

  40. Villatoro, D., Sen, S., & Sabater-Mir, J. (2010). Of social norms and sanctioning: A game theoretical overview. International Journal of Agent Technologies and Systems, 2, 1–15.

    Article  Google Scholar 

  41. Vu, T., Powers, R., & Shoham, Y. (2006). Learning against multiple opponents. In AAMAS ’06: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems (pp. 752–759). New York, NY, USA: ACM.

  42. Walker, A., & Wooldridge, M. (1995). Understanding the emergence of conventions in multi-agent systems. In Proceedings of the First International Conference on Multi-Agent Systems (ICMAS95) (pp. 384–389). San Francisco.

  43. Watkins, C. J. C. H., & Dayan, P. (1992). Q-learning. Machine Learning, 8(3–4), 279–292.

    MATH  Google Scholar 

  44. Wolpert, D. H., & Tumer, K. (2002). Collective intelligence, data routing and Braess’ paradox. Journal of Artificial Intelligence Research, 16, 359–387.

    MATH  MathSciNet  Google Scholar 

  45. Young, P. H. (1993). The evolution of conventions. Econometrica, 61(1), 57–84.

    Article  MATH  MathSciNet  Google Scholar 

  46. Young, P. H. (1996). The economics of convention. The Journal of Economic Perspectives, 10(2), 105–122.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. LAMSADE, Université Paris Dauphine, Place du Maréchal de Lattre de Tassigny, 75775 , Paris Cedex 16, France

    Stéphane Airiau

  2. Department of Mathematical & Computer Sciences, The University of Tulsa, 800 South Tucker Drive, Tulsa, OK, 74104, USA

    Sandip Sen

  3. IIIA - Artificial Intelligence Research Institute, CSIC - Spanish Scientific Research Council, Campus Universitat Autonoma de Barcelona, Bellaterra, 08193 , Catalonia, Spain

    Daniel Villatoro

Authors
  1. Stéphane Airiau
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sandip Sen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Villatoro
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Stéphane Airiau.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Airiau, S., Sen, S. & Villatoro, D. Emergence of conventions through social learning. Auton Agent Multi-Agent Syst 28, 779–804 (2014). https://doi.org/10.1007/s10458-013-9237-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10458-013-9237-x

Keywords

Search

Navigation