Abstract
Reactive or dynamic planning is currently the dominant paradigm for controlling virtual agents in 3D videogames. Various reactive planning techniques are employed in the videogame industry while many reactive planning systems and languages are being developed in the academia. Claims about benefits of different approaches are supported by the experience of videogame programmers and the arguments of researchers, but rigorous empirical data corroborating alleged advantages of different methods are lacking. Here, we present results of a pilot study in which we compare the usability of an academic technique designed for programming intelligent agents’ behavior with the usability of an unaltered classical programming language. Our study seeks to replicate the situation of professional game programmers considering using an unfamiliar academic system for programming in-game agents. We engaged 30 computer science students attending a university course on virtual agents in two programming assignments. For each, the students had to code high-level behavior of a 3D virtual agent solving a game-like task in the Unreal Tournament 2004 environment. Each student had to use Java for one task and the POSH reactive planner with a graphical editor for the other. We collected quantitative and qualitative usability data. The results indicate that POSH outperforms Java in terms of usability for one of the assigned tasks but not the other. This implies that the suitability of an AI systems-engineering approach is task sensitive. We also discuss lessons learnt about the evaluation process itself, proposing possible improvements in the experimental design. We conclude that comparative studies are a useful method for analyzing benefits of different approaches to controlling virtual agents.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Fu, D., Houlette, R.: The Ultimate Guide to FSMs in Games. In: AI Game Programming Wisdom II, pp. 283–302. Charles River Media (2004)
Champandard, A.J.: Behavior Trees for Next-Gen Game AI. Internet presentation (January 18, 2011), http://aigamedev.com/insider/presentations/behavior-trees
Schuytema, P.: Game Development with Lua. Charles River Media (2005)
UnrealScript programming language (January 18, 2011), http://unreal.epicgames.com/UnrealScript.htm
Schwab, B.: AI Game Engine Programming, 2nd edn. Charles River Media (2008)
AiGameDev community (January 18, 2011), http://aigamedev.com/
Rabin S.: AI Game Programming Wisdom series (January 18, 2011), http://www.aiwisdom.com/
Gamasutra webpage (January 18, 2011), http://www.gamasutra.com/
Magerko, B., Laird, J.E., Assanie, M., Kerfoot, A., Stokes, D.: AI Characters and Directors for Interactive Computer Games. In: Proceedings of the 2004 Innovative Applications of Artificial Intelligence Conference, San Jose, CA. AAAI Press (July 2004)
Best, B.J., Lebiere, C.: Cognitive agents interacting in real and virtual worlds. In: Sun, R. (ed.) Cognition and Multi-Agent Interaction: From Cognitive Modeling to Social Simulation. Cambridge University Press, NY, NY (2006)
Hindriks, K.V., van Riemsdijk, B., Behrens, T., Korstanje, R., Kraayenbrink, N., Pasman, W., de Rijk, L.: Unreal Goal Bots: Conceptual Design of a Reusable Interface. In: Dignum, F. (ed.) Agents for Games and Simulations II. LNCS (LNAI), vol. 6525, pp. 1–18. Springer, Heidelberg (2011)
Bryson, J.J.: Inteligence by design: Principles of Modularity and Coordination for Engineering Complex Adaptive Agent. PhD Thesis, MIT, Department of EECS, Cambridge, MA (2001)
Partington, S.J., Bryson, J.J.: The Behavior Oriented Design of an Unreal Tournament Character. In: Panayiotopoulos, T., Gratch, J., Aylett, R.S., Ballin, D., Olivier, P., Rist, T. (eds.) IVA 2005. LNCS (LNAI), vol. 3661, pp. 466–477. Springer, Heidelberg (2005)
Köster, M., Novák, P., Mainzer, D., Fuhrmann, B.: Two Case Studies for Jazzyk BSM. In: Dignum, F., Bradshaw, J., Silverman, B., van Doesburg, W. (eds.) Agents for Games and Simulations. LNCS (LNAI), vol. 5920, pp. 33–47. Springer, Heidelberg (2009)
Dignum, F., Bradshaw, J., Silverman, B., van Doesburg, W. (eds.): Agents for Games and Simulations. LNCS, vol. 5920. Springer, Heidelberg (2009)
Tyrrell, T.: Computational Mechanisms for Action Selection. Ph.D. Dissertation. Centre for Cognitive Science, University of Edinburgh (1993)
Bryson, J.J.: Hierarchy and Sequence vs. Full Parallelism in Action Selection. In: Simulation of Adaptive Behavior 6, Paris, pp. 147–156 (2000)
Bryson, J.J.: Action Selection and Individuation in Agent Based Modelling. In: Proceedings of Agent 2003: Challenges of Social Simulation, Argonne National Laboratory, pp. 317–330 (2003)
Hindriks, K.V., van Riemsdijk, M.B., Jonker, C.M.: An Empirical Study of Patterns in Agent Programs. In: Desai, N., Liu, A., Winikoff, M. (eds.) PRIMA 2010. LNCS, vol. 7057, pp. 196–211. Springer, Heidelberg (2012)
Brom, C.: Curricula of the course on modelling behaviour of human and animal-like agents. In: Proceedings of the Frontiers in Science Education Research Conference, Famagusta, North, Cyprus (2009)
Gemrot, J., Brom, C., Kadlec, R., Bída, M., Burkert, O., Zemčák, M., Píbil, R., Plch, T.: Pogamut 3 – Virtual Humans Made Simple. In: Gray, J. (ed.) Advances in Cognitive Science, pp. 211–243. The Institution Of Engineering And Technology (2010)
Brom, C., Gemrot, J., Burkert, O., Kadlec, R., Bída, M.: 3D Immersion in Virtual Agents Education. In: Spierling, U., Szilas, N. (eds.) ICIDS 2008. LNCS, vol. 5334, pp. 59–70. Springer, Heidelberg (2008)
Artifical beings course, practical lessons slides (January 18, 2011), http://diana.ms.mff.cuni.cz/pogamut-devel/doku.php?id=lectures
Pogamut 3 platform documentation (January 25, 2011), http://diana.ms.mff.cuni.cz/main/tiki-index.php?page=Documentation
Artificial beings course, final exam package (January 18, 2011), http://diana.ms.mff.cuni.cz/pogamut-devel/doku.php?id=human-like_artifical_agents_2009-10_summer_semester_exam_info
Bordini, R.H., Hübner, J.F., Wooldridge, M.: Programming Multi-Agent Systems in AgentSpeak Using Jason. John Wiley & Sons, Ltd. (2007)
Brooks, R.A.: Intelligence Without Representation. Artificial Intelligence 47(1-3), 139–159 (1991)
Bozada, T.A., Perkins, T.K., North, M.J., Kathy, K.L., Simunich, L., Tatara, E.: An Applied Approach to Representing Human Behavior in Military Logistics Operations. In: Fall Simulation Interoperability Workshop, Simulation Standards Interoperability Organization, Orlando, FL USA (September 2006)
Desai, N.: Using Describers To Simplify ScriptEase. Master Thesis. Department of Computing Science, University of Alberta, Edmonton, Alberta, Canada (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gemrot, J., Brom, C., Bryson, J., Bída, M. (2012). How to Compare Usability of Techniques for the Specification of Virtual Agents’ Behavior? An Experimental Pilot Study with Human Subjects. In: Beer, M., Brom, C., Dignum, F., Soo, VW. (eds) Agents for Educational Games and Simulations. AEGS 2011. Lecture Notes in Computer Science(), vol 7471. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-32326-3_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-32326-3_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-32325-6
Online ISBN: 978-3-642-32326-3
eBook Packages: Computer ScienceComputer Science (R0)