Abstract
Evolutionary Algorithms (EAs) are frequently used as a mechanism for the optimization of autonomous agents in games (bots), but knowing when to stop the evolution, when the bots are good enough, is not as easy as it would a priori seem. The first issue is that optimal bots are either unknown (and thus unusable as termination condition) or unreachable. In most EAs trying to find optimal bots fitness is evaluated through game playing. Many times it is found to be noisy, making its use as a termination condition also complicated. A fixed amount of evaluations or, in the case of games, a certain level of victories does not guarantee an optimal result. Thus the main objective of this paper is to test several termination conditions in order to find the one that yields optimal solutions within a restricted amount of time, and that allows researchers to compare different EAs as fairly as possible. To achieve this we will examine several ways of finishing an EA who is finding an optimal bot design process for a particular game, Planet Wars in this case, with the characteristics described above, determining the capabilities of every one of them and, eventually, selecting one for future designs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
The ‘1’ in all denominators is used to avoid dividing by 0 and for the ratio calculation.
- 3.
This number has been used in order to leverage the computational power of a cluster with this number of nodes.
References
Small, R., Bates-Congdon, C.: Agent Smith: Towards an evolutionary rule-based agent for interactive dynamic games. In: IEEE Congress on Evolutionary Computation, CEC 2009, pp. 660–666 (2009)
Mora, A.M., Montoya, R., Merelo, J.J., Sánchez, P.G., Castillo, P.A., Laredo, J.L.J., Martínez, A.I., Espacia, A.: Evolving bot AI in unreal\(^{\rm TM}\). In: Di Chio, C., et al. (eds.) EvoApplicatons 2010, Part I. LNCS, vol. 6024, pp. 171–180. Springer, Heidelberg (2010)
Esparcia-Alcazár, A.I., Martínez-García, A., Mora, A.M., Merelo, J.J., García-Sánchez, P.: Genetic evolution of fuzzy finite state machines to control bots in a first-person shooter game. In: GECCO 2010: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 829–830. ACM (2010)
Fernández-Ares, A., García-Sánchez, P., Mora, A.M., Guervós, J.J.M.: Adaptive bots for real-time strategy games via map characterization. In: CIG, pp. 417–721. IEEE (2012)
Bäck, T.: Evolutionary algorithms in theory and practice. Oxford University Press, New York (1996)
Roche, D., Gil, D., Giraldo, J.: Detecting loss of diversity for an efficient termination of eas. In: 15th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing, SYNASC 2013, Timisoara, Romania, 23–26 September, pp. 561–566. IEEE (2013)
Merelo, J.J., Castillo, P.A., Mora, A., Fernández-Ares, A., Esparcia-Alcázar, A.I., Cotta, C., Rico, N.: Studying and tackling noisy fitness in evolutionary design of game characters. In: Rosa, A., Merelo, J.J., Filipe, J. (eds.) ECTA 2014 - Proceedings of the International Conference on Evolutionary Computation Theory and Applications, pp. 76–85 (2014)
Mora, A.M., Fernández-Ares, A., Guervós, J.J.M., García-Sánchez, P., Fernandes, C.M.: Effect of noisy fitness in real-time strategy games player behaviour optimisation using evolutionary algorithms. J. CST 27(5), 1007–1023 (2012)
Fernández-Ares, A., Mora, A.M., Guervós, J.J.M., García-Sánchez, P., Fernandes, C.: Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: IEEE Conference on Evolutionary Computation, pp. 2017–2024. IEEE (2011)
Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)
García-Sánchez, P., Fernández-Ares, A., Mora, A.M., Castillo, P.A., González, J., Guerv, J.J.M.: Tree depth influence in genetic programmingfor generation of competitive agentsfor RTS games. In: Esparcia-Alcázar, A.I., Mora, A.M. (eds.) EvoApplications 2014. LNCS, vol. 8602, pp. 409–419. Springer, Heidelberg (2014)
Lara-Cabrera, R., Cotta, C., Fernández-Leiva, A.: On balance and dynamism in procedural content generation with self-adaptive evolutionary algorithms. Nat. Comput. 13(2), 157–168 (2014)
Nogueira-Collazo, M., Fernández-Leiva, A.: Virtual player design using self-learning via competitive coevolutionary algorithms. Nat. Comput. 13(2), 131–144 (2014)
Jin, Y., Branke, J.: Evolutionary optimization in uncertain environments - a survey. IEEE Trans. Evol. Comput. 9(3), 303–317 (2005)
Qian, C., Yu, Y., Zhou, Z.H.: Analyzing evolutionary optimization in noisy environments. CoRR: abs/1311.4987 (2013). http://arxiv.org/abs/1311.4987
Branke, J.: Creating robust solutions by means of evolutionary algorithms. In: Eiben, A.E., Bäck, T., Schoenauer, M., Schwefel, H.-P. (eds.) PPSN 1998. LNCS, vol. 1498, pp. 119–128. Springer, Heidelberg (1998)
Branke, J.: Evolutionary optimization in dynamic environments, pp. 125–172. Kluwer Academic Publisher (2001)
Markon, S., Arnold, D., Bäck, T., Beielstein, T., Beyer, H.: Thresholding - a selection operator for noisy ES. In: Kim, J.-H., Zhang, B.-T., Fogel, G., Kuscu, I. (eds.) Proceedings of 2001 Congress on Evolutionary Computation (CEC 2001), pp. 465–472. IEEE Press, Piscataway (2001)
Bhattacharya, M., Islam, R., Mahmood, A.: Uncertainty and evolutionary optimization: A novel approach. CoRR abs/1407.4000 (2014)
Goel, T., Stander, N.: A non-dominance-based online stopping criterion for multi-objective evolutionary algorithms. Int. J. Numer. Meth. Eng. 84(6), 661–684 (2010)
Wagner, T., Trautmann, H.: Online convergence detection for evolutionary multi-objective algorithms revisited. In: Fogel, G., Ishibuchi, H. (eds.) Proceedings of International Congress on Evolutionary Computation (CEC 2010), pp. 3554–3561. IEEE press (2010)
Safe, M., Carballido, J.A., Ponzoni, I., Brignole, N.B.: On stopping criteria for genetic algorithms. In: Bazzan, A.L.C., Labidi, S. (eds.) SBIA 2004. LNCS (LNAI), vol. 3171, pp. 405–413. Springer, Heidelberg (2004)
Rudolph, G.: Convergence analysis of canonical genetic algorithms. IEEE Trans. Neural Netw. 5, 96–101 (1994)
Hart, W., Baden, S., Belew, R., Kohn, S.: Analysis of the numerical effects of parallelism on a parallel genetic algorithm. In: Proceedings of the 10th International Parallel Processing Symposium, pp. 606–612. IEEE Computer Society (1996)
Sena, G., Megherbi, D., Isern, G.: Implementation of a parallel genetic algorithm on a cluster of workstations: travelling salesman problem, a case study. Future Gener. Comput. Syst. 17, 477–488 (2001)
Esparcia-Alcázar, A.I., Moravec, J.: Fitness approximation for bot evolution in genetic programming. Soft Comput. 17(8), 1479–1487 (2013)
Harper, R.: Evolving robocode tanks for Evo robocode. Genet. Programm. Evolvable Mach. 15(4), 403–431 (2014)
Fernández Leiva, A.J., O’Valle Barragán, J.L.: Decision tree-based algorithms for implementing bot AI in UT2004. In: Ferrández, J.M., Álvarez Sánchez, J.R., de la Paz, F., Toledo, F.J. (eds.) IWINAC 2011, Part I. LNCS, vol. 6686, pp. 383–392. Springer, Heidelberg (2011)
Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1(1), 67–82 (1997)
Acknowledgments
This work has been supported in part by SIPESCA (Programa Operativo FEDER de Andalucía 2007–2013), TIN2011-28627-C04-02 (Spanish Ministry of Economy and Competitivity), SPIP2014-01437 (Dirección General de Tráfico), PRY142/14 (Fundación Pública Andaluza Centro de Estudios Andaluces en la IX Convocatoria de Proyectos de Investigación) and PYR-2014-17 GENIL project (CEI-BIOTIC Granada).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Fernández-Ares, A. et al. (2015). It’s Time to Stop: A Comparison of Termination Conditions in the Evolution of Game Bots. In: Mora, A., Squillero, G. (eds) Applications of Evolutionary Computation. EvoApplications 2015. Lecture Notes in Computer Science(), vol 9028. Springer, Cham. https://doi.org/10.1007/978-3-319-16549-3_29
Download citation
DOI: https://doi.org/10.1007/978-3-319-16549-3_29
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16548-6
Online ISBN: 978-3-319-16549-3
eBook Packages: Computer ScienceComputer Science (R0)