Abstract
A virtual world is an online community in the form of a computer-based simulated environment, through which users can interact with one another and use and create objects. The non-player characters (NPC) in virtual world are following a fixed set of pre-programmed behaviors and lack the ability to adapt with the changing surrounding. Reinforcement learning agent is a way to deal with this problem. However, in a cooperative social environment, NPC should learn not only by trial and error, but also through cooperation by sharing information. The key investigation of this paper is: modeling the NPCs as multi-agent, and enable them to conduct cooperative learning, then speeding up the learning process. By using a fire fighting scenario in Robocup Rescue, our research shows that sharing information between cooperative agents will outperform independent agents who do not communicate during learning. The further work and some important issues of multi-agent reinforcement learning in virtual world will also be discussed in this paper.
Keywords
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bishop, J.: Enhancing the understanding of genres of web-based communities: The role of the ecological cognition framework. International Journal of Web-Based Communities 5(1), 4–17 (2009) (Available online)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research, 237–285 (1996)
Shoham, Y., Powers, R., Grenager, T.: Multi Agent Reinforcement Learning: a critical survey (2003)
Lauer, M., Riedmiller, M.: An algorithm for distributed reinforcement learning in cooperative multi-agent systems. In: Proceedings Seventeenth International Conference on Machine Learning (ICML-2000), Stanford University, US, 29 June – 2 July, pp. 535–542 (2000)
Busoniu, L., Babuska, R., De Schutter, B.: Multi Agent Reinforcement Learning: A survey. In: Proceedings of the 9th International Conference on Control, Automation, Robotics and Vision (ICARCV 2006), Singapore, pp. 527–532 (2006)
Tan, M.: Multi Agent Reinforcement Learning Independent vs Cooperative Agents. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 330–337. Morgan Kaufmann, San Francisco (1993)
Merrick, K., Maher, M.L.: Motivated Reinforcement Learning for Non-Player Characters in Persistent Computer Game Worlds. In: Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology (2006)
Watkins, C.J.C.H.: Learning from Delayed Rewards. PhD thesis, Cambridge University, Cambridge, England (1989)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Zhang, P., Ma, X., Pan, Z., Li, X., Xie, K. (2010). Multi-Agent Cooperative Reinforcement Learning in 3D Virtual World. In: Tan, Y., Shi, Y., Tan, K.C. (eds) Advances in Swarm Intelligence. ICSI 2010. Lecture Notes in Computer Science, vol 6145. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13495-1_90
Download citation
DOI: https://doi.org/10.1007/978-3-642-13495-1_90
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13494-4
Online ISBN: 978-3-642-13495-1
eBook Packages: Computer ScienceComputer Science (R0)