Article

Motivated reinforcement learning for non-player characters in persistent computer game worlds

Authors:

Kathryn Merrick,

Mary Lou MaherAuthors Info & Claims

ACE '06: Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology

Pages 3 - es

https://doi.org/10.1145/1178823.1178828

Published: 14 June 2006 Publication History

Abstract

Massively multiplayer online computer games are played in complex, persistent virtual worlds. Over time, the landscape of these worlds evolves and changes as players create and personalise their own virtual property. In contrast, many non-player characters that populate virtual game worlds possess a fixed set of pre-programmed behaviours and lack the ability to adapt and evolve in time with their surroundings. This paper presents motivated reinforcement learning agents as a means of creating non-player characters that can both evolve and adapt. Motivated reinforcement learning agents explore their environment and learn new behaviours in response to interesting experiences, allowing them to display progressively evolving behavioural patterns. In dynamic worlds, environmental changes provide an additional source of interesting experiences triggering further learning and allowing the agents to adapt their existing behavioural patterns in time with their surroundings.

References

[1]

Berlyne, D. E. Aesthetics and psychobiology. Englewood Cliffs, NJ: Prentice-Hall, 1971.]]

[2]

Canamero, L. Modelling motivations and emotions as a basis for intelligent behaviour. In Proceedings of the First International Symposium on Autonomous Agents. (New York, NY), ACM Press, 1995, 148--155]]

Digital Library

[3]

Graepel, T., Herbrich, R., and Gold, J. Learning to Fight. Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design and Education. 2004.]]

[4]

Kohonen, T. Self-organisation and associative memory. Springer, (Berlin), 1993.]]

Digital Library

[5]

Laird, J., and van Lent, M. Interactive computer games: human-level AI's killer application. In Proceedings of AAAI National Conference on Artificial Intelligence, 2000, 1171--1178.]]

Digital Library

[6]

Maher, M.-L., and Gero, J. S. Agent models of 3D virtual worlds, ACADIA 2002: Thresholds. (California State Polytechnic University, Pamona), 2002, 127--138.]]

[7]

Marsland, S., Nehmzow, U. and Shapiro, J. A real-time novelty detector for a mobile robot. EUREL European Advanced Robotics Systems Masterclass and Conference, 2000.]]

[8]

Merceron, A. Languages and Logic. Pearson Education Australia, 2001.]]

[9]

Nilsson, N., Introduction to machine learning, http://ai.stanford.edu/people/nilsson/mlbook.html, Accessed April 2006.]]

[10]

Saunders, R., & Gero, J. S. Curious agents and situated design evaluations. In J. S. Gero & F. M. T. Brazier (Eds.), Agents In Design, 2002, 133--149.]]

[11]

Siege University, 303: Skrit, http://garage.gaspowered.com, Accessed March, 2006.]]

[12]

Stanley, J. C., Computer simulation of a model of habituation, Nature, 261:146--148, 1976.]]

[13]

Sutton, S. and Barto, A. Reinforcement learning: an introduction. The MIT Press, 2000.]]

Digital Library

[14]

Watkins, C., Learning from delayed rewards, PhD Thesis, Cambridge University, (Cambridge, England), 1989.]]

[15]

Woodcock, S., Games making interesting use of artificial intelligence techniques, http://www.gameai.com/ Accessed March, 2006.]]

Cited By

Greally M(2023)Enhancing Safety and Collaboration in Human Robot Interaction for Industrial RoboticsJournal of Robotics Spectrum10.53759/9852/JRS202301013(134-143)Online publication date: 22-Nov-2023
https://doi.org/10.53759/9852/JRS202301013
Li JCai SYang QHuang H(2023)How to Enrich Metaverse? Blockchains, AI, and Digital TwinFrom Blockchain to Web3 & Metaverse10.1007/978-981-99-3648-9_2(27-61)Online publication date: 25-May-2023
https://doi.org/10.1007/978-981-99-3648-9_2
Yang QZhao YHuang HXiong ZKang JZheng Z(2022)Fusing Blockchain and AI With Metaverse: A SurveyIEEE Open Journal of the Computer Society10.1109/OJCS.2022.31882493(122-136)Online publication date: 2022
https://doi.org/10.1109/OJCS.2022.3188249
Show More Cited By

Index Terms

Motivated reinforcement learning for non-player characters in persistent computer game worlds
1. Computing methodologies
  1. Artificial intelligence
    1. Search methodologies
      1. Heuristic function construction
  2. Machine learning
    1. Machine learning approaches
      1. Neural networks
2. Theory of computation
  1. Design and analysis of algorithms
    1. Algorithm design techniques
      1. Dynamic programming

Recommendations

Motivated reinforcement learning for adaptive characters in open-ended simulation games
ACE '07: Proceedings of the international conference on Advances in computer entertainment technology

Recently a new generation of virtual worlds has emerged in which users are provided with open-ended modelling tools with which they can create and modify world content. The result is evolving virtual spaces for commerce, education and social ...
Modeling motivation for adaptive nonplayer characters in dynamic computer game worlds
Theoretical and Practical Computer Applications in Entertainment

Current computer games are being set in increasingly more complex and dynamic virtual environments. Massively multiplayer online games, for example, are played in persistent virtual worlds, which evolve and change as players create and personalize their ...
A survey of students' computer game playing habits

The popularity of computer games has attracted the attention of educationalists who are interested in finding out whether the features that make them so engaging could be captured and used to help people learn more effectively. In this paper we examine ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ACE '06: Proceedings of the 2006 ACM SIGCHI international conference on Advances in computer entertainment technology

June 2006

572 pages

ISBN:1595933808

DOI:10.1145/1178823

General Chairs:
Hiroshi Ishii
Tangible Media Group, MIT Media Laboratory
,
Newton Lee
Association for Computing Machinery (ACM), Disney Online, Woodbury University, and Bell Labs
,
Stephane Natkin
CNAM Paris, CEDRIC
,
Katsuhide Tsushima
Osaka Electro-Communication University, Japan

Copyright © 2006 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 14 June 2006

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Article

Conference

ACE2006

Sponsor:

SIGCHI

ACE2006: International Conference in Advances in Computer Entertainment Technology

June 14 - 16, 2006

California, Hollywood, USA

Acceptance Rates

Overall Acceptance Rate 36 of 90 submissions, 40%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

35
Total Citations
View Citations
1,119
Total Downloads

Downloads (Last 12 months)30
Downloads (Last 6 weeks)2

Reflects downloads up to 02 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Greally M(2023)Enhancing Safety and Collaboration in Human Robot Interaction for Industrial RoboticsJournal of Robotics Spectrum10.53759/9852/JRS202301013(134-143)Online publication date: 22-Nov-2023
https://doi.org/10.53759/9852/JRS202301013
Li JCai SYang QHuang H(2023)How to Enrich Metaverse? Blockchains, AI, and Digital TwinFrom Blockchain to Web3 & Metaverse10.1007/978-981-99-3648-9_2(27-61)Online publication date: 25-May-2023
https://doi.org/10.1007/978-981-99-3648-9_2
Yang QZhao YHuang HXiong ZKang JZheng Z(2022)Fusing Blockchain and AI With Metaverse: A SurveyIEEE Open Journal of the Computer Society10.1109/OJCS.2022.31882493(122-136)Online publication date: 2022
https://doi.org/10.1109/OJCS.2022.3188249
Futami KTerada TTsukamoto M(2021)A Method for Behavior Change Support by Controlling Psychological Effects on Walking Motivation Caused by Step Count Log Competition SystemSensors10.3390/s2123801621:23(8016)Online publication date: 30-Nov-2021
https://doi.org/10.3390/s21238016
Makri SCharalambous P(2021)Towards a multi-agent non-player character road network: a Reinforcement Learning approach2021 IEEE Conference on Games (CoG)10.1109/CoG52621.2021.9619047(1-5)Online publication date: 17-Aug-2021
https://doi.org/10.1109/CoG52621.2021.9619047
Schroeder BFraulini NVan Buskirk WJohnson C(2020)Using a Non-player Character to Improve Training Outcomes for Submarine Electronic Warfare OperatorsAdaptive Instructional Systems10.1007/978-3-030-50788-6_39(531-542)Online publication date: 10-Jul-2020
https://doi.org/10.1007/978-3-030-50788-6_39
Roohi STakatalo JGuckelsberger CHämäläinen PMandryk RHancock MPerry MCox A(2018)Review of Intrinsic Motivation in Simulation-based Game TestingProceedings of the 2018 CHI Conference on Human Factors in Computing Systems10.1145/3173574.3173921(1-13)Online publication date: 21-Apr-2018
https://dl.acm.org/doi/10.1145/3173574.3173921
Petrovic V(2018)Artificial Intelligence and Virtual Worlds – Toward Human-Level AI AgentsIEEE Access10.1109/ACCESS.2018.28559706(39976-39988)Online publication date: 2018
https://doi.org/10.1109/ACCESS.2018.2855970
Prasetio YWijaya RSjah MChristian MChowanda A(2017)Location-based game to enhance players experience in survival horror gameProcedia Computer Science10.1016/j.procs.2017.10.037116:C(206-213)Online publication date: 1-Nov-2017
https://dl.acm.org/doi/10.1016/j.procs.2017.10.037
Razzaq SMaqbool FKhalid MTariq IZahoor AIlyas M(2017)Zombies Arena: fusion of reinforcement learning with augmented reality on NPCCluster Computing10.1007/s10586-017-0969-221:1(655-666)Online publication date: 13-Jun-2017
https://doi.org/10.1007/s10586-017-0969-2
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten