Abstract
In this paper, a hierarchical learning framework is proposed for emergence of social norms in networked multiagent systems. This framework features a bottom level of agents and several levels of supervisors. Agents in the bottom level interact with each other using reinforcement learning methods, and report their information to their supervisors after each interaction. Supervisors then aggregate the reported information and produce guide policies by exchanging information with other supervisors. The guide policies are then passed down to the subordinate agents in order to adjust their learning behaviors heuristically. Experiments are carried out to explore the efficiency of norm emergence under the proposed framework, and results verify that learning from local interactions integrating hierarchical supervision can be an effective mechanism for emergence of social norms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Savarimuthu, B.T.R., Arulanandam, R., Purvis, M.: Aspects of active norm learning and the effect of lying on norm emergence in agent societies. In: Kinny, D., Hsu, J.Y., Governatori, G., Ghose, A.K. (eds.) PRIMA 2011. LNCS, vol. 7047, pp. 36–50. Springer, Heidelberg (2011)
Mukherjee, P., Sen, S., Airiau, S.: Norm emergence under constrained interactions in diverse societies. In: Proceedings of 7th AAMAS, pp. 779–786 (2008)
Hao, J., Sun, J., Huang, D., Cai, Y., Yu, C.: Heuristic collective learning for efficient and robust emergence of social norms. In: Proceedings of 14th AAMAS, pp. 1647–1648 (2015)
Yu, C., Zhang, M., Ren, F.: Collective learning for the emergence of social norms in networked multiagent systems. IEEE Trans. Cybernet. 44(12), 2342–2355 (2014)
Yu, C., Zhang, M., Ren, F., Luo, X.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of the 12th AAMAS, pp. 475–482 (2013)
Yu, C., Zhang, M., Ren, F., Hao, J.: Emergence of social norms through collective learning in networked agent societies. In: Proceedings of MFSC@AAMAS 2014 (2014)
Villatoro, D., Sabater-Mir, J., Sen, S.: Social instruments for robust convention emergence. In: Proceedings of 22nd IJCAI, pp. 420–425 (2011)
Shoham, Y., Tennenholtz, M.: On the emergence of social conventions: modeling, analysis, and simulations. Artif. Intell. 94(1), 139–166 (1997)
Young, H.P.: The economics of convention. J. Econ. Perspect. 10(2), 105–122 (1996)
Sen, S., Airiau, S.: Emergence of norms through social learning. In: Proceedings of the 20nd IJCAI, pp. 1507–1512 (2007)
Sen, O., Sen, S.: Effects of social network topology and options on norm emergence. In: Padget, J., Artikis, A., Vasconcelos, W., Stathis, K., da Silva, V.T., Matson, E., Polleres, A. (eds.) COIN@AAMAS 2009. LNCS, vol. 6069, pp. 211–222. Springer, Heidelberg (2010)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An introduction. MIT Press, Cambridge (1998)
Weibull, J.W.: Evolutionary Game Theory. MIT Press, Cambridge (1997)
Pacheco, J.M., Traulsen, A., Nowak, M.A.: Coevolution of strategy and structure in complex networks with dynamical linking. Phys. Rev. Lett. 97(25), 1–9 (2006)
Bowling, M., Veloso, M.: Multiagent learning using a variable learning rate. Artif. Intell. 136(2), 215–250 (2002)
Savarimuthu, B.: Norm Learning in Multi-agent Societies. University of Otago, Dunedin (2011)
Airiau, S., Sen, S., Villatoro, D.: Emergence of conventions through social learning. Auton. Agent. Multi-Agent Syst. 28(5), 779–804 (2014)
Villatoro, D., Sen, S., Sabater-Mir. J.: Topology and memory effect on convention emergence. In: Proceedings of the 2009 WI and IAT, pp. 233–240 (2009)
Villatoro, D., Sabater-Mir, J., Sen, S.: Robust convention emergence in social networks through self-reinforcing structures dissolution. ACM Trans. Auton. Adapt. Syst. 8(1), 2–21 (2013)
Mihaylov, M., Tuyls, K., Now, A.: A decentralized approach for convention emergence in multi-agent systems. Auton. Agent. Multi-Agent Syst. 15(2), 1–30 (2013)
Mahmoud, S., Griffiths, N., Keppens, J.: Norm emergence: overcoming hub effects in scale free networks. In: Proceedings of COIN, pp. 136–150 (2012)
Zhang, C., Sherief, A., Lesser, V.: Integrating organizational control into multi-agent learning. In: Proceedings of the 7th AAMAS, pp. 757–764 (2009)
Bianchi, R.A.C., Ribeiro, C.H.C., Costa, A.H.R.: Heuristic selection of actions in multiagent reinforcement learning. In: Proceedings of 2007 IJCAI, pp. 690–695 (2007)
Acknowledgments
This work is supported by the National Natural Science Foundation of China under Grant 61502072, Fundamental Research Funds for the Central Universities of China under Grant DUT14RC(3)064, and Post-Doctoral Science Foundation of China under Grants 2014M561229 and 2015T80251.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Yu, C., Lv, H., Ren, F., Bao, H., Hao, J. (2015). Hierarchical Learning for Emergence of Social Norms in Networked Multiagent Systems. In: Pfahringer, B., Renz, J. (eds) AI 2015: Advances in Artificial Intelligence. AI 2015. Lecture Notes in Computer Science(), vol 9457. Springer, Cham. https://doi.org/10.1007/978-3-319-26350-2_56
Download citation
DOI: https://doi.org/10.1007/978-3-319-26350-2_56
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-26349-6
Online ISBN: 978-3-319-26350-2
eBook Packages: Computer ScienceComputer Science (R0)