Kathryn Merrick
ABSTRACT
Massively multiplayer online computer games are played in complex, persistent virtual worlds. Over time, the landscape of these worlds evolves and changes as players create and personalise their own virtual property. In contrast, many non-player characters that populate virtual game worlds possess a fixed set of pre-programmed behaviours and lack the ability to adapt and evolve in time with their surroundings. This paper presents motivated reinforcement learning agents as a means of creating non-player characters that can both evolve and adapt. Motivated reinforcement learning agents explore their environment and learn new behaviours in response to interesting experiences, allowing them to display progressively evolving behavioural patterns. In dynamic worlds, environmental changes provide an additional source of interesting experiences triggering further learning and allowing the agents to adapt their existing behavioural patterns in time with their surroundings.
- Berlyne, D. E. Aesthetics and psychobiology. Englewood Cliffs, NJ: Prentice-Hall, 1971.]]Google Scholar
- Canamero, L. Modelling motivations and emotions as a basis for intelligent behaviour. In Proceedings of the First International Symposium on Autonomous Agents. (New York, NY), ACM Press, 1995, 148--155]] Google ScholarDigital Library
- Graepel, T., Herbrich, R., and Gold, J. Learning to Fight. Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design and Education. 2004.]]Google Scholar
- Kohonen, T. Self-organisation and associative memory. Springer, (Berlin), 1993.]] Google ScholarDigital Library
- Laird, J., and van Lent, M. Interactive computer games: human-level AI's killer application. In Proceedings of AAAI National Conference on Artificial Intelligence, 2000, 1171--1178.]] Google ScholarDigital Library
- Maher, M.-L., and Gero, J. S. Agent models of 3D virtual worlds, ACADIA 2002: Thresholds. (California State Polytechnic University, Pamona), 2002, 127--138.]]Google Scholar
- Marsland, S., Nehmzow, U. and Shapiro, J. A real-time novelty detector for a mobile robot. EUREL European Advanced Robotics Systems Masterclass and Conference, 2000.]]Google Scholar
- Merceron, A. Languages and Logic. Pearson Education Australia, 2001.]]Google Scholar
- Nilsson, N., Introduction to machine learning, http://ai.stanford.edu/people/nilsson/mlbook.html, Accessed April 2006.]]Google Scholar
- Saunders, R., & Gero, J. S. Curious agents and situated design evaluations. In J. S. Gero & F. M. T. Brazier (Eds.), Agents In Design, 2002, 133--149.]]Google Scholar
- Siege University, 303: Skrit, http://garage.gaspowered.com, Accessed March, 2006.]]Google Scholar
- Stanley, J. C., Computer simulation of a model of habituation, Nature, 261:146--148, 1976.]]Google ScholarCross Ref
- Sutton, S. and Barto, A. Reinforcement learning: an introduction. The MIT Press, 2000.]] Google ScholarDigital Library
- Watkins, C., Learning from delayed rewards, PhD Thesis, Cambridge University, (Cambridge, England), 1989.]]Google Scholar
- Woodcock, S., Games making interesting use of artificial intelligence techniques, http://www.gameai.com/ Accessed March, 2006.]]Google Scholar
Index Terms
- Motivated reinforcement learning for non-player characters in persistent computer game worlds
Recommendations
Motivated reinforcement learning for adaptive characters in open-ended simulation games
ACE '07: Proceedings of the international conference on Advances in computer entertainment technologyRecently a new generation of virtual worlds has emerged in which users are provided with open-ended modelling tools with which they can create and modify world content. The result is evolving virtual spaces for commerce, education and social ...
Modeling motivation for adaptive nonplayer characters in dynamic computer game worlds
Theoretical and Practical Computer Applications in EntertainmentCurrent computer games are being set in increasingly more complex and dynamic virtual environments. Massively multiplayer online games, for example, are played in persistent virtual worlds, which evolve and change as players create and personalize their ...
A survey of students' computer game playing habits
The popularity of computer games has attracted the attention of educationalists who are interested in finding out whether the features that make them so engaging could be captured and used to help people learn more effectively. In this paper we examine ...
Comments