Abstract
One of the most overlooked challenges in artificial intelligence (AI) for computer games is to create non-player game characters (NPCs) with human-like behavior. Modern NPCs determine their actions in different situations using certain decision-making methods, enabling them to change the current state of the game world. In this paper, we survey current decision-making methods used by NPCs in games, identifying five categories. We give detailed overview of these five categories and determine the previous studies that belong to each of these categories. We also discuss the hybrid methods which are the combinations of different decision-making methods and the frameworks that are created for NPC decision-making. As a result of this analysis, we create a taxonomy table based on these covered studies. Lastly, the challenges faced in our study and future possibilities for improvement are described.
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig1_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig2_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig3_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig4_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig5_HTML.png)
![](http://media.springernature.com/m312/springer-static/image/art%3A10.1007%2Fs10462-023-10491-7/MediaObjects/10462_2023_10491_Fig6_HTML.png)
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Abiyev RH, Günsel I, Akkaya N et al (2016) Robot soccer control using behaviour trees and fuzzy logic. Procedia Comput Sci 102(August):477–484. https://doi.org/10.1016/j.procs.2016.09.430
Belle S, Gittens C, Graham TN (2022) A framework for creating non-player characters that make psychologically-driven decisions. In: 2022 IEEE international conference on consumer electronics (ICCE), IEEE. pp 1–7. https://doi.org/10.1109/ICCE53296.2022.9730383
Bourg DM, Seemann G (2004) AI for game developers. O’Reilly Media Inc, Sebastopol
Cavazza M (2000) Al in computer games: survey and perspectives. Virtual Real 5(4):223–235. https://doi.org/10.1007/BF01408521
Charles D, McGlinchey S (2004) The past, present and future of artificial neural networks in digital games. In: Proceedings of the 5th international conference on computer games: artificial intelligence, design and education. The University of Wolverhampton, pp 163–169
Colledanchise M, Ögren P (2018) Behavior trees in robotics and AI: an introduction. CRC Press, Cambridge
Colledanchise M, Marzinotto A, Ögren P (2014) Performance analysis of stochastic behavior trees. In: 2014 IEEE international conference on robotics and automation (ICRA), IEEE. pp 3265–3272. https://doi.org/10.1109/ICRA.2014.6907328
da Silva GA, de Souza Ribeiro MW (2021) Development of non-player character with Believable Behavior: a systematic literature review. In: Anais Estendidos do XX Simpósio Brasileiro de Jogos e Entretenimento Digital. SBC, Porto Alegre, pp 319–323. https://doi.org/10.5753/sbgames_estendido.2021.19660
Danielson M, Ekenberg L (2011) Decision making in intelligent agents. In: Sierra AP, Rabunal JR, Dorado J (eds) Encyclopedia of artificial intelligence. IGI Global, Pennsylvania, pp 431–436
Dell’Acqua P, Costantini S (2022) Emotional behavior trees for empathetic human-automation interaction. In: WOA 2022: 23rd workshop from objects to agents
Dey R, Child C (2013) QL-BT: Enhancing behaviour tree design and implementation with Q-learning. In: 2013 IEEE conference on computational inteligence in games (CIG), IEEE. pp 1–8. https://doi.org/10.1109/CIG.2013.6633623
Dias J, Mascarenhas S, Paiva A (2014) FAtiMA modular: towards an agent architecture with a generic appraisal framework. In: Bosse T, Broekens J, Dias J et al (eds) Emotion modeling: towards pragmatic computational models of affective processes. Springer, Cham, pp 44–56
El-Nasr MS, Yen J, Ioerger TR (2000) FLAME—fuzzy logic adaptive model of emotions. Auton Agent Multi-Agent Syst 3(3):219–257. https://doi.org/10.1023/A:1010030809960
Fathoni K, Hakkun R, Nurhadi H (2020) Finite state machines for building believable non-playable character in the game of Khalid ibn Al-Walid. J Phys Conf Ser 1577:012018. https://doi.org/10.1088/1742-6596/1577/1/012018
Fauzi R, Hariadi M, Nugroho SMS et al (2019) Defense behavior of real time strategy games: comparison between hfsm and fsm. Indones J Electr Eng Comput Sci 13(2):634–642. https://doi.org/10.11591/ijeecs.v13.i2.pp634-642
Florez-Puga G, Gomez-Martin M, Diaz-Agudo B et al (2008) Dynamic expansion of behaviour trees. In: Proceedings of the AAAI conference on artificial intelligence and interactive digital entertainment. pp 36–41. https://doi.org/10.1609/aiide.v4i1.18669
Folkman S, Lazarus RS (1984) Stress, appraisal, and coping. Springer, New York
Forgy CL (1979) On the efficient implementation of production systems. PhD thesis, Carnegie Mellon University
Forgy CL (1989) Rete: a fast algorithm for the many pattern/many object pattern match problem. In: Mylopolous J, Brodie M (eds) Readings in artificial intelligence and databases. Morgan Kaufmann, San Francisco, pp 547–559
Fu D, Houlette R (2002) Putting AI in entertainment: an AI authoring tool for simulation and games. IEEE Intell Syst 17(4):81–84. https://doi.org/10.1109/MIS.2002.1024756
Fu Y, Qin L, Yin Q (2016) A reinforcement learning behavior tree framework for game AI. In: 2016 international conference on economics, social science, arts, education and management engineering. Atlantis Press, pp 573–579
Fujii S, Nakashima T, Ishibuchi H (2008) A study on constructing fuzzy systems for high-level decision making in a car racing game. In: 2008 IEEE congress on evolutionary computation, CEC 2008, IEEE. pp 3626–3633
Georgeson J, Child C (2016) NPCs as people, too: the extreme AI personality engine. https://doi.org/10.48550/arXiv.1609.04879
Guimaraes M, Santos P, Jhala A (2017) CiF-CK: an architecture for social NPCs in commercial games. In: 2017 IEEE conference on computational intelligence and games (CIG), IEEE. pp 126–133. https://doi.org/10.1109/CIG.2017.8080425
Gupta N (2013) Artificial neural network. Netw Complex Syst 3(1):24–28
Harel D (1987) Statecharts: a visual formalism for complex systems. Sci Comput Prog 8(3):231–274. https://doi.org/10.1016/0167-6423(87)90035-9
Harper PR (2005) A review and comparison of classification algorithms for medical decision making. Health Policy 71(3):315–331. https://doi.org/10.1016/j.healthpol.2004.05.002
Holba J, Huber G (2021) Open-world enemy AI in Mafia III. In: Game AI pro—online edition 2021. Game AI pro, chap 16
Iovino M, Scukins E, Styrud J et al (2022) A survey of behavior trees in robotics and AI. Robot Auton Syst 154(104):096. https://doi.org/10.1016/j.robot.2022.104096
Isla D (2005) Gdc 2005 proceeding: handling complexity in the halo 2 AI. In: Game developers conference 2005. https://www.gamedeveloper.com/programming/gdc-2005-proceeding-handling-complexity-in-the-i-halo-2-i-ai
Johansen NS, Kær LB, Stolberg JAB, et al. (2022) Towards believable non-player characters with domain-independent planning. In: 2022 workshop on scheduling and planning applications workshop
Johansson A, Dell’Acqua P (2012) Comparing behavior trees and emotional behavior networks for npcs. In: 2012 17th international conference on computer games (CGAMES), IEEE. pp 253–260. https://doi.org/10.1109/CGames.2012.6314584
Johnson D, Wiles J (2001) Computer games with intelligence. In: 10th IEEE international conference on fuzzy systems. (Cat. No. 01CH37297), IEEE. pp 1355–1358. https://doi.org/10.1109/FUZZ.2001.1008909
Jolly K, Ravindran K, Vijayakumar R et al (2007) Intelligent decision making in multi-agent robot soccer system through compounded artificial neural networks. Robot Auton Syst 55(7):589–596. https://doi.org/10.1016/j.robot.2006.12.011
Justesen N, Bontrager P, Togelius J et al (2020) Deep learning for video game playing. IEEE Trans Games 12(1):1–20. https://doi.org/10.1109/TG.2019.2896986
Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285. https://doi.org/10.1613/jair.301
Kitchenham B (2004) Procedures for performing systematic reviews, vol 33. Keele University, Keele, pp 1–26
Kopel M, Hajas T (2018) Implementing AI for non-player characters in 3D video games. In: Nguyen NT, Hoang DH, Hong TP et al (eds) Intelligent information and database systems. Springer, Cham, pp 610–619
Laird JE (2001) It knows what you’re going to do: adding anticipation to a quakebot. In: Proceedings of the fifth international conference on autonomous agents. pp 385–392. https://doi.org/10.1145/375735.376343
Lara-Cabrera R, Nogueira-Collazo M, Cotta C et al (2015) Game artificial intelligence: challenges for the scientific community. In: CoSECivi 2015. https://doi.org/10630/9942
Li Y, Musilek P, Wyard-Scott L (2004) Fuzzy logic in agent-based game design. In: IEEE annual meeting of the fuzzy information, 2004. Processing NAFIPS’04, IEEE. pp 734–739. https://doi.org/10.1109/NAFIPS.2004.1337393
Lie CSK, Istiono W (2022) How to make npc learn the strategy in fighting games using adaptive AI? Int J Sci Tech Res Eng 7:4
Lin J, He J, Zhang N (2019) Application of behavior tree in AI design of MOBA games. In: 2019 IEEE 2nd international conference on knowledge innovation and invention (ICKII), IEEE. pp 323–326. https://doi.org/10.1109/ICKII46306.2019.9042660
Long E (2007) Enhanced NPC behaviour using goal oriented action planning. Master’s thesis, University of Abertay Dundee, Dundee
Loyall BA, Bates J (1991) Hap a reactive, adaptive architecture for agents. In: Citeseer
Madsen CABCW, Adamatti DF (2013) Using artificial neural networks in NPC decision-making process. Int J Comput Inf Technol 2(6):1009–1013
Mas’udi NA, Jonemaro EMA, Akbar MA et al (2021) Development of non-player character for 3d kart racing game using decision tree. Fount Inform J 6(2):51–60. https://doi.org/10.21111/fij.v6i2.4678
Mascarenhas S, Guimarães M, Santos PA et al (2021) FAtiMA Toolkit—toward an effective and accessible tool for the development of intelligent virtual agents and social robots. Preprint at http://arxiv.org/abs/2103.03020
Mateas M, Stern A (2002) A behavior language for story-based believable agents. IEEE Intell Syst 17(4):39–47. https://doi.org/10.1109/MIS.2002.1024751
McCoy J, Treanor M, Samuel B et al (2010) Comme il Faut 2: a fully realized model for socially-oriented gameplay. In: Proceedings of the intelligent narrative technologies III workshop. Association for Computing Machinery, New York, pp 1–8. https://doi.org/10.1145/1822309.1822319
McCoy J, Treanor M, Samuel B et al (2011) Comme il Faut: a system for authoring playable social models. In: Proceedings of the AAAI conference on artificial intelligence and interactive digital entertainment
McCoy J, Treanor M, Samuel B et al (2014) Social story worlds with Comme il Faut. IEEE Trans Comput Intell AI Games 6(2):97–112. https://doi.org/10.1109/TCIAIG.2014.2304692
Meng F, Hyung CJ (2022) Research on multi-npc marine game ai system based on q-learning algorithm. In: 2022 IEEE international conference on artificial intelligence and computer applications (ICAICA), IEEE. pp 648–652. https://doi.org/10.1109/ICAICA54878.2022.9844648
Millington I (2019) Artificial intelligence for games, 3rd edn. CRC Press, Cambridge
Miyake Y, Shirakami Y, Shimokawa K et al (2019) A character decision-making system for FINAL FANTASY XV by combining behavior trees and state machines. In: Game AI pro 360: guide to architecture. CRC Press, p 339. https://doi.org/10.4324/9781315151700
Moher D, Liberati A, Tetzlaff J et al (2009) Preferred reporting items for systematic reviews and meta-analyses: the prisma statement. Ann Intern Med 151(4):264–269
Nareyek A (2000) Intelligent agents for computer games. In: International conference on computers and games. Springer, pp 414–422. https://doi.org/10.1007/3-540-45579-5_28
Neufeld X, Mostaghim S, Sancho-Pradel DL et al (2019) Building a planner: a survey of planning systems used in commercial video games. IEEE Trans Games 11(2):91–108. https://doi.org/10.1109/TG.2017.2782846
Niewiadomski A, Renkas K (2014) Hierarchical fuzzy logic systems and controlling vehicles in computer games. J Appl Comput Sci 22(1):201–212
Novák V, Perfilieva I, Mockor J (1999) Mathematical principles of fuzzy logic, vol 517. Springer, New York
OBrien L (1996) Fuzzy logic in games. Game Dev Mag 3(2):52–55
Ochs M, Sabouret N, Corruble V (2009) Simulation of the dynamics of nonplayer characters’ emotions and social relations in games. IEEE Trans Comput Intell AI Games 1(4):281–297. https://doi.org/10.1109/TCIAIG.2009.2036247
Ohsone K, Onisawa T (2008) Friendly partner system of poker game with facial expressions. In: 2008 IEEE symposium on computational intelligence and games, IEEE. pp 95–102. https://doi.org/10.1109/CIG.2008.5035626
Orkin J (2003) Applying goal-oriented action planning to games. In: Rabin S (ed) AI game programming wisdom, vol 2. Charles River Media Inc, New York
Orkin J (2006) Three states and a plan: the AI of FEAR. In: Game developers conference 2006. CMP Game Group, San Jose, p 4
Panwar H (2022) The NPC AI of the last of us: a case study. Preprint at https://doi.org/10.48550/ARXIV.2207.00682
Partlan N, Soto L, Howe J et al (2022) EvolvingBehavior: towards co-creative evolution of behavior trees for game NPCs. In: Proceedings of the 17th international conference on the foundations of digital games. Association for Computing Machinery, New York. https://doi.org/10.1145/3555858.3555896
Perez D, Nicolau M, O’Neill M et al (2011) Evolving behaviour trees for the mario AI competition using grammatical evolution. In: Kaufmann P, Castillo PA (eds) Applications of evolutionary computation. Springer, Berlin, pp 123–132
Perrie J, Li L (2014) Building a dynamic social community with non playable characters. IEICE Trans Inf Syst 97(8):1965–1973. https://doi.org/10.1587/transinf.E97.D.1965
Pirovano M (2012) The use of fuzzy logic for artificial intelligence in games. University of Milano, Milan
Pittman DL (2007) Practical development of goal-oriented action planning AI. Master’s thesis, Southern Methodist University, Dallas
Popescu A, Broekens J, Van Someren M (2013) GAMYGDALA: an emotion engine for games. IEEE Trans Affect Comput 5(1):32–44. https://doi.org/10.1109/T-AFFC.2013.24
Pyjas GM, Weinel J, Broadhead M (2022) Storytelling and vr: inducing emotions through AI characters. In: Proc EVA London 2022. pp 198–204. https://doi.org/10.14236/ewic/EVA2022.37
Quadir AM, Khder MA (2022) Exploring the potential of AI-driven opponents in video games. In: 2022 ASU international conference in emerging technologies for sustainability and intelligent systems (ICETSIS), IEEE. pp 464–469. https://doi.org/10.1109/ICETSIS55481.2022.9888909
Quinlan JR (1986) Induction of decision trees. Mach Learn 1(1):81–106. https://doi.org/10.1007/BF00116251
Receveur JB, Victor S, Melchior P (2020) Autonomous car decision making and trajectory tracking based on genetic algorithms and fractional potential fields. Intel Serv Robot 13(2):315–330. https://doi.org/10.1007/s11370-020-00314-x
Rodrigues S, Rayat HK, Kurichithanam RM et al (2021) Shriek: a role playing game using unreal engine 4 and behaviour trees. In: 2021 4th biennial international conference on nascent technologies in engineering (ICNTE), IEEE. pp 1–6. https://doi.org/10.1109/ICNTE51185.2021.9487723
Rothmann S (2003) The big five personality dimensions and job performance. SA J Ind Psychol 29(1):68–74. https://doi.org/10.10520/EJC88938
Russell S, Norvig P (2002) Artificial intelligence: a modern approach. Prentice Hall, Hoboken
Sagredo-Olivenza I, Gómez-Martín PP, Gómez-Martín MA et al (2017) Trained behavior trees: programming by demonstration to support AI game designers. IEEE Trans Games 11(1):5–14. https://doi.org/10.1109/TG.2017.2771831
Schwab P, Hlavacs H (2015) Capturing the essence: towards the automated generation of transparent behavior models. In: Proceedings of the AAAI conference on artificial intelligence and interactive digital entertainment
Sekhavat YA (2017) Behavior trees for computer games. Int J Artif Intell Tools 26(02):1730001. https://doi.org/10.1142/S0218213017300010
Shaout A, King BW, Reisner LA (2006) Real-time game design of Pac-Man using fuzzy logic. Int Arab J Inf Technol 3(4):315–325
Shu F, Chaudhari NS (2008) A chaotic behavior decision algorithm based on self-generating neural network for computer games. In: 2008 3rd IEEE conference on industrial electronics and applications, IEEE. pp 1912–1915. https://doi.org/10.1109/ICIEA.2008.4582852
Sielicki M, Daszuta M, Szajerman D (2018) Adaptation and application of goal oriented action planning in unreal engine. Comput Game Innov 1:103
Simonov A, Zagarskikh A, Fedorov V (2019) Applying behavior characteristics to decision-making process to create believable game AI. Procedia Comput Sci 156:404–413. https://doi.org/10.1016/j.procs.2019.08.222
Sindhu RM, Annabel LSP, Monisha G (2022) Development of a 2d game using artificial intelligence in unity. In: 2022 6th international conference on trends in electronics and informatics (ICOEI), IEEE. pp 1031–1037. https://doi.org/10.1109/ICOEI53556.2022.9776750
Sloan C (2015) Drive-based utility-maximizing computer game non-player characters. PhD thesis, Technological University Dublin, Dublin. https://doi.org/10.21427/D7KK57
Sloan C, Mac Namee B, Kelleher JD (2011) Utility-directed goal-oriented action planning: a utility-based control system for computer game agents. University of Ulster, MartinMcGinnity Intelligent Systems Research Centre, Coleraine
Soylucicek AE, Bostanci E, Safak AB (2017) A fuzzy logic based attack strategy design for enemy drones in meteor escape game. Int J Comput Theory Eng 9(3):167–171. https://doi.org/10.7763/ijcte.2017.v9.1132
Steunebrink BR, Dastani M, Meyer JJC (2009) The occ model revisited. In: Proc 4th workshop on emotion and computing. Association for the Advancement of Artificial Intelligence
Studiawan R, Hariadi M (2018) Tactical planning in space game using goal-oriented action planning. J Adv Res Electr Eng 2:1. https://doi.org/10.12962/j25796216.v2.i1.32
Suyikno DA, Setiawan A (2019) Feasible NPC hiding behaviour using goal oriented action planning in case of hide-and-seek 3D game simulation. In: 2019 fourth international conference on informatics and computing (ICIC), IEEE. pp 1–6, https://doi.org/10.1109/ICIC47613.2019.8985962
Sweetser P, Wiles J (2002) Current AI in games: a review. Austral J Intell Inf Process Syst 8(1):24–42
Syahputra M, Arippa A, Rahmat R et al (2019) Historical theme game using finite state machine for actor behaviour. J Phys Conf Ser 1:012122. https://doi.org/10.1088/1742-6596/1235/1/012122
Van Waveren JMP (2001) The quake III arena bot. Master’s thesis. Delft University of Technology, Delft
West N (1996) Next generation lexicon A to Z: a definitive guide to gaming terminology. NEXT generation. https://archive.org/details/nextgen-issue-015/page/n39/mode/2up. Accessed 21 Nov 2022
Wexler J (2002) Artificial intelligence in games. University of Rochester, Rochester
Widhiyasana Y, Harika M, Hakim FFN et al (2022) Genetic algorithm for artificial neural networks in real-time strategy games. Int J Inform Vis 6(2):298–305. https://doi.org/10.30630/joiv.6.2.832
Yanase J, Triantaphyllou E (2019) The seven key challenges for the future of computer-aided diagnosis in medicine. Int J Med Inform 129:413–422. https://doi.org/10.1016/j.ijmedinf.2019.06.017
Yannakakis GN, Togelius J (2018) Artificial intelligence and games. Springer, New York
Yue B, de Byl P (2006) The state of the art in game AI standardisation. In: Proceedings of the 2006 international conference on game research and development. Murdoch University, Murdoch, pp 41–46. https://doi.org/10.5555/1234341.1234350
Zarozinski M, Than L (2001) Imploding combinatorial explosion in a fuzzy system. In: AI game programming wisdom, Vol. 2. Charles River Media, Inc., pp 342–350
Zhu M, Feng L (2022) Design and implementation of NPC AI based on genetic algorithm and BP neural network. In: Proceedings of the 14th international conference on computer modeling and simulation. Association for Computing Machinery, New York, pp 168–173. https://doi.org/10.1145/3547578.3547604
Zijie W, Tongyu W, Hang G (2021) A survey: development and application of behavior trees. In: Liang Q, Wang W, Liu X et al (eds) Communications, signal processing, and systems. Springer, Singapore, pp 1581–1589
Acknowledgements
The authors thank the anonymous reviewers for their valuable comments and feedback which have improved the manuscript significantly. They also thank Simon Edward Mumford for his help in language editing and proofreading.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Uludağlı, M.Ç., Oğuz, K. Non-player character decision-making in computer games. Artif Intell Rev 56, 14159–14191 (2023). https://doi.org/10.1007/s10462-023-10491-7
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-023-10491-7