Abstract
Choosing actions within norm-regulated environments involves balancing achieving one’s goals and coping with any penalties for non-compliant behaviour. This choice becomes more complicated in environments where there is uncertainty. In this paper, we address the question of choosing actions in environments where there is uncertainty regarding both the outcomes of agent actions and the intensity of monitoring for norm violations. Our technique assumes no prior knowledge of probabilities over action outcomes or the likelihood of norm violations being detected by employing reinforcement learning to discover both the dynamics of the environment and the effectiveness of the enforcer. Results indicate agents become aware of greater rewards for violations when enforcement is lax, which gradually become less attractive as the enforcement is increased.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsNotes
- 1.
A city foreign to the agent’s designer.
- 2.
In a slight abuse of notation, we shall denote by \(\mathcal {D}(n)\) the detection probability of the violation of the norm \(n\in \mathcal {N}\) where \(n\) is constant at all time points t.
References
Alechina, N., Dastani, M., Logan, B.: Norm approximation for imperfect monitors. In: Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems, AAMAS, pp. 117–124 (2014)
Beheshti, R., Sukthankar, G.: A normative agent-based model for predicting smoking cessation trends. In: Proceedings of the International Conference on Autonomous Agents and Multi-Agent Systems, pp. 557–564 (2014)
Cliffe, O., De Vos, M., Padget, J.: Specifying and reasoning about multiple institutions. In: Noriega, P., Vázquez-Salceda, J., Boella, G., Boissier, O., Dignum, V., Fornara, N., Matson, E. (eds.) COIN 2006. LNCS (LNAI), vol. 4386, pp. 67–85. Springer, Heidelberg (2007)
Dastani, M., Meyer, J.-J.C., Grossi, D.: A logic for normative multi-agent programs. J. Log. Comput. 23(2), 335–354 (2013)
Esteva, M., de la Cruz, D., Sierra, C.: ISLANDER: an electronic institutions editor. In: Proceedings of the First International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS 2002, pp. 1045–1052. ACM, New York (2002)
Fagundes, M.S.: Sequential Decision Making in Normative Environments. Ph.D. thesis, Universidad Rey Juan Carlos (2012)
Fagundes, M.S., Billhardt, H., Ossowski, S.: Reasoning about norm compliance with rational agents. In: Coelho, H., Studer, R., Wooldridge, M. (eds.) ECAI. Frontiers in Artificial Intelligence and Applications, vol. 215, pp. 1027–1028. IOS Press (2010)
Fagundes, M.S., Ossowski, S., Luck, M., Miles, S.: Using normative markov decision processes for evaluating electronic contracts. AI Commun. 25(1), 1–17 (2012)
Hübner, J.F., Sichman, J.S., Boissier, O.: Developing organised multiagent systems using the \({\cal M}\)OISE\(^{+}\) model: programming issues at the system and agent levels. Int. J. Agent-Oriented Softw. Eng. 1(3/4), 370–395 (2007)
Kollingbaum, M.J., Norman, T.J.: Norm adoption and consistency in the NoA agent architecture. In: Dastani, M., Dix, J., El Fallah-Seghrouchni, A. (eds.) PROMAS 2003. LNCS (LNAI), vol. 3067, pp. 169–186. Springer, Heidelberg (2004)
Meneguzzi, F., Logan, B., Fagundes, M.S.: Norm monitoring with asymmetric information. In: Proceedings of the Thirteenth International Conference on Autonomous Agents and Multiagent Systems, pp. 1523–1524 (2014)
Meneguzzi, F., Luck, M.: Norm-based behaviour modification in BDI agents. In: Proceedings of the International Conference on Autonomous Agents and Multiagent Systems, pp. 177–184 (2009)
Morales, J., Lopez-Sanchez, M., Rodriguez-Aguilar, J.A., Wooldridge, M., Vasconcelos, W.: Automated synthesis of normative systems. In: Proceedings of the International Conference on Autonomous agents and Multi-agent systems, pp. 483–490 (2013)
Rummery, G.A., Niranjan, M.: On-line q-learning using connectionist systems. Technical report TR 166, Cambridge University Engineering Department (1994)
Russell, S.J., Norvig, P.: Artificial Intelligence - A Modern Approach, 3rd edn. Pearson Education, Upper Saddle River (2010)
Savarimuthu, B.T.R., Cranefield, S.: Norm creation, spreading and emergence: a survey of simulation models of norms in multi-agent systems. Multiagent Grid Syst. 7(1), 21–54 (2011)
Savarimuthu, B.T.R., Cranefield, S., Purvis, M.A., Purvis, M.K.: Obligation norm identification in agent societies. J. Artif. Soc. Soc. Simul. 13, 4 (2010)
Savarimuthu, B.T.R., Cranefield, S., Purvis, M.A., Purvis, M.K.: Identifying conditional norms in multi-agent societies. In: De Vos, M., Fornara, N., Pitt, J.V., Vouros, G. (eds.) COIN 2010. LNCS, vol. 6541, pp. 285–302. Springer, Heidelberg (2011)
Watkins, C.: Learning from Delayed Rewards. Ph.D. thesis, King’s College Cambridge (1989)
Yan-bin, P., Gao, J., Ai, J.-Q., Wang, C.-H., Hang, G.: An extended agent BDI model with norms, policies and contracts. In: 4th International Conference on Wireless Communications, Networking and Mobile Computing, pp. 1–4, October 2008
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Li, J., Meneguzzi, F., Fagundes, M., Logan, B. (2016). Reinforcement Learning of Normative Monitoring Intensities. In: Dignum, V., Noriega, P., Sensoy, M., Sichman, J. (eds) Coordination, Organizations, Institutions, and Norms in Agent Systems XI. COIN 2015. Lecture Notes in Computer Science(), vol 9628. Springer, Cham. https://doi.org/10.1007/978-3-319-42691-4_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-42691-4_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-42690-7
Online ISBN: 978-3-319-42691-4
eBook Packages: Computer ScienceComputer Science (R0)