Abstract
Despite the recent results on formalizing multiagent reinforcement learning using stochastic games, the exponential increase of the space of joint actions prevents the use of this formalism in systems of many agents. In fact, most of the literature concentrates on repeated games with single state and few joint actions. However, many real-world systems are comprised of a much higher number of agents. Also, these are normally not homogeneous and interact in environments which are highly dynamic. This paper discusses the implications of co-evolution between two classes of agents in stochastic games using learning automata. These agents interact in a urban traffic scenario where approaches based on the standard stochastic games are prohibitive. The approach was tested in a network with different traffic conditions.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Balmer, M., Nagel, K., Raney, B.: Large–scale multi-agent simulations for transportation applications. Journal of Intelligent Transportation Systems: Technology, Planning, and Operations 8(4), 205–221 (2004)
Bazzan, A.L.C.: A distributed approach for coordination of traffic signal agents. Autonomous Agents and Multiagent Systems 10(1), 131–164 (2005)
Bottom, J., Ben-Akiva, M., Bierlaire, M., Chabini, I.: Generation of consistent anticipatory route guidance. In: Proceedings of TRISTAN III, San Juan, Puerto Rico, vol. 2 (June 1998)
Bowling, M.H., Veloso, M.M.: Rational and convergent learning in stochastic games. In: Nebel, B. (ed.) Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, Seattle, pp. 1021–1026. Morgan Kaufmann, San Francisco (2001)
Camponogara, E., Kraus Jr., W.: Distributed learning agents in urban traffic control. In: Pires, F.M., Abreu, S.P. (eds.) EPIA 2003. LNCS (LNAI), vol. 2902, pp. 324–335. Springer, Heidelberg (2003)
Claus, C., Boutilier, C.: The dynamics of reinforcement learning in cooperative multiagent systems. In: Proceedings of the Fifteenth National Conference on Artificial Intelligence, pp. 746–752 (1998)
Hu, J., Wellman, M.P.: Multiagent reinforcement learning: Theoretical framework and an algorithm. In: Proc. 15th International Conf. on Machine Learning, pp. 242–250. Morgan Kaufmann, San Francisco (1998)
Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the 11th International Conference on Machine Learning ML, New Brunswick, NJ, pp. 157–163. Morgan Kaufmann, San Francisco (1994)
Liu, R., Van Vliet, D., Watling, D.: Microsimulation models incorporating both demand and supply dynamics. Transportation Research Part A: Policy and Practice 40(2), 125–150 (2006)
Nunes, L., Oliveira, E.C.: Learning from multiple sources. In: Jennings, N., Sierra, C., Sonenberg, L., Tambe, M. (eds.) Proceedings of the 3rd International Joint Conference on Autonomous Agents and Multi Agent Systems, AAMAS, vol. 3, pp. 1106–1113. IEEE Computer Society, New York (2004)
Papageorgiou, M.: Traffic control. In: Hall, R.W. (ed.) Handbook of Transportation Science, vol. 8, pp. 243–277. Kluwer Academic Pub., Dordrecht (2003)
Shoham, Y., Powers, R., Grenager, T.: If multi-agent learning is the answer, what is the question? Artificial Intelligence 171(7), 365–377 (2007)
Stone, P.: Multiagent learning is not the answer. It is the question. Artificial Intelligence 171(7), 402–405 (2007)
Tumer, K., Wolpert, D.: A survey of collectives. In: Tumer, K., Wolpert, D. (eds.) Collectives and the Design of Complex Systems, pp. 1–42. Springer, Heidelberg (2004)
van Zuylen, H.J., Taale, H.: Urban networks with ring roads: a two-level, three player game. In: Proc. of the 83rd Annual Meeting of the Transportation Research Board. TRB (January 2004)
Verbeeck, K., Nowé, A., Peeters, M., Tuyls, K.: Multi-agent reinforcement learning in stochastic games and multi-stage games. In: Kudenko, D., Kazakov, D., Alonso, E. (eds.) Adaptive Agents and Multi-Agent Systems II. LNCS (LNAI), vol. 3394, pp. 275–294. Springer, Heidelberg (2005)
Vu, T., Powers, R., Shoham, Y.: Learning against multiple opponents. In: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, pp. 752–760 (2006)
Wardrop, J.G.: Some theoretical aspects of road traffic research. Proceedings of the Institute of Civil Engineers 2, 325–378 (1952)
Wiering, M.: Multi-agent reinforcement learning for traffic light control. In: ICML 2000. Proceedings of the Seventeenth International Conference on Machine Learning, pp. 1151–1158 (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bazzan, A.L.C., Klügl, F., Nagel, K. (2007). Adaptation in Games with Many Co-evolving Agents. In: Neves, J., Santos, M.F., Machado, J.M. (eds) Progress in Artificial Intelligence. EPIA 2007. Lecture Notes in Computer Science(), vol 4874. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-77002-2_17
Download citation
DOI: https://doi.org/10.1007/978-3-540-77002-2_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-77000-8
Online ISBN: 978-3-540-77002-2
eBook Packages: Computer ScienceComputer Science (R0)