Skip to main content

Advertisement

SpringerLink
Log in

Creating autonomous agents for playing Super Mario Bros game by means of evolutionary finite state machines

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

This paper shows the design and improvement of an autonomous agent based in using evolutionary methods to improve behavioural models (finite state machines), which are part of the individuals to evolve. This leads to the obtention of a so-called bot that follows the Gameplay track rules of the international Mario AI Championship and is able to autonomously complete different scenarios on a simulator of Super Mario Bros. game. Mono- and multi-seed approaches (evaluation in one play or multiple plays respectively) have been analysed, in order to compare respectively the performance of an approach focused in solving a specific scenario, and another more general, devoted to obtain an agent which can play successfully in different scenarios. The analysis considers the machine resources consumption, which turns in a bottleneck in some experiments. However, the methods yield agents which can finish several stages of different difficulty levels, and playing much better than an expert human player, since they can deal with very difficult situations (several enemies surrounding Mario, for instance) in real time. According to the results and considering the competition’s restrictions (time limitations) and objectives (complete scenarios up to difficulty level 3), these agents have enough performance to participate in this competition track.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. Designer and producer of Nintendo Ltd., and winner of the 2012 Príncipe de Asturias Prize in Humanities and Communication.

  2. http://www.mojang.com/notch/mario/.

  3. http://www.marioai.org/.

  4. http://www.marioai.org/.

References

  1. Bäck T (1996) Evolutionary algorithms in theory and practice. Oxford University Press, Oxford

    MATH  Google Scholar 

  2. Baumgarten R. Mario AI A* agent. http://www.doc.ic.ac.uk/rb1006/projects:marioai

  3. Bojarski S, Bates-Congdon C (2011) REALM: a rule-based evolutionary computation agent that learns to play Mario. In: Proceedings of the 2011 IEEE symposium on computational intelligence and games (CIG 2011), IEEE Press, pp 83–90

  4. Booth TL (1967) Sequential machines and automata theory, 1st edn. Wiley, New York

    MATH  Google Scholar 

  5. Esparcia-Alcázar AI, Martínez-García AI, Mora AM, Merelo JJ, García-Sánchez P (2010) Controlling bots in a first person shooter game using genetic algorithms. In: Proceedings of the 2010 IEEE congress on evolutionary computation (CEC 2010), pp 1–8

  6. Esparcia-Alcázar AI, Martínez-García AI, Mora AM, Merelo JJ, García-Sánchez P (2010) Genetic evolution of fuzzy finite state machines to control bots in a first-person shooter game. In: Proceedings of the GECCO 2010, pp 829–830

  7. Fernández AJ, González-Jiménez J (2005) Action games: evolutive experiences. In: Computational intelligence, theory and applications advances in soft computing, vol 33, pp 487–501

  8. Fernández-Ares A, Mora AM, Merelo JJ, García-Sánchez P, Fernandes C (2011) Optimizing player behavior in a real-time strategy game using evolutionary algorithms. In: Proceedings of the 2011 IEEE congress on evolutionary computation, CEC ’11, pp 2017–2024

  9. Fu D, Houlette R (2004) The ultimate guide to FSMs in games. AI Game Programming Wisdom 2, Charles River Media, pp 283–302

  10. Goldberg DE, Korb B, Deb K (1989) Messy genetic algorithms: motivation, analysis, and first results. Complex Syst 3(5):493–530

    MATH  MathSciNet  Google Scholar 

  11. Hagelbäck J (2012) Potential-field based navigation in StarCraft. In: Proceedings of the 2012 IEEE symposium on computational intelligence and games (CIG 2012), IEEE Press, pp 388–393

  12. Jang SH, Yoon JW, Cho SB (2009) Optimal strategy selection of non-player character on real time strategy game using a speciated evolutionary algorithm. In: Proceedings of the 5th IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 75–79

  13. Karakovskiy S, Togelius J (2012) The Mario AI benchmark and competitions. IEEE Trans Comput Intell AI Games 4(1):55–67

    Google Scholar 

  14. Laird JE (2001) Using a computer game to develop advanced AI. Computer 34(7):70–75

    Article  Google Scholar 

  15. Martín E, Martínez M, Recio G, Saez Y (2010) Pac-mAnt: optimization based on ant colonies applied to developing an agent for Ms. Pac-Man. In: Proceedings of the 2010 IEEE conference on computational intelligence and games (CIG 2010), IEEE Press, pp 458–464

  16. Mora AM, Montoya R, Merelo JJ, García-Sánchez P, Castillo PA, Laredo JLJ, Martínez A, Esparcia AI (2010) Evolving bot AI in unreal. In: CDC et al (eds) Applications of evolutionary computing, part I. Vol 6024 of LNCS., Springer, New York, pp 170–179

  17. Mora AM, Moreno MA, Merelo JJ, Castillo PA, García-Arenas MI, Laredo JLJ (2010) Evolving the cooperative behaviour in UnrealTM bots. In: Proceedings of the 2010 IEEE conference on computational intelligence and games (CIG 2010), IEEE Press, pp 241–248

  18. Mora AM, Fernández-Ares A, Merelo JJ, García-Sánchez P (2012) Dealing with noisy fitness in the design of a RTS game bot. In: Proceedings of the applications of evolutionary computing: EvoApplications 2012, LNCS, vol 7248, Springer, New York, pp 234–244

  19. Nareyek A (2004) AI in computer games. ACM Queue J 1(10):58–65

    Google Scholar 

  20. Pedersen C, Togelius J, Yannakakis G (2009) Modeling player experience in Super Mario Bros. In: Proceedings of the 2009 IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 132–139

  21. Pepels T, Windans HM (2012) Enhancements for Monte-Carlo Tree Search in Ms Pac-Man. In: Proceedings of the 2012 IEEE conference on computational intelligence and games (CIG 2012), IEEE Press, pp 265–272

  22. Ponsen M, Munoz-Avila H, Spronck P, Aha DW (2006) Automatically generating game tactics through evolutionary learning. AI Mag 27(3):75–84

    Google Scholar 

  23. Shaker N, Togelius J, Yannakakis GN, Weber B, Shimizu T, Hashiyama T, Sorenson N, Pasquier P, Mawhorter P, Takahashi G, Smith G, Baumgarten R (2011) The 2010 Mario AI championship: level generation track. IEEE Trans Comput Intell AI Games 3(4):332–347

    Google Scholar 

  24. Shaker N, Nicolau M, Yannakakis GN, Togelius J, O’neill M (2012) Evolving levels for Super Mario Bros using grammatical evolution. In: Proceedings of the 2012 IEEE symposium on computational intelligence and games (CIG 2012), IEEE Press, pp 304–311

  25. Shaker N, Togelius J, Yannakakis G, Poovanna L, Ethiraj VS, Johansson SJ, Reynolds RG, Heether LK, Schumann T, Gallagher M (2013) The Turing test track of the 2012 Mario AI championship: entries and evaluation. In: Proceedings of the 2013 IEEE symposium on computational intelligence and games (CIG 2013), IEEE Press, pp 1–8

  26. Small R, Bates-Congdon C (2009) Agent Smith: towards an evolutionary rule-based agent for interactive dynamic games. In: Proceedings of the 2009 IEEE congress on evolutionary computation (CEC’09), pp 660–666

  27. Spronck P, Sprinkhuizen-Kuyper I, Postma E (2003) Improving opponent intelligence through offline evolutionary learning. Int J Intell Games Simul 2(1):20–27

    Google Scholar 

  28. Synnaeve G, Bessière P (2011) A Bayesian model for RTS units control applied to StarCraft. In: Proceedings of the 2011 IEEE symposium on computational intelligence and games (CIG 2011), IEEE Press, pp 190–196

  29. Togelius J, Karakovskiy S, Koutnik J, Schmidhuber J (2009) Super Mario evolution. In: Proceedings of the 2009 IEEE symposium on computational intelligence and games (CIG’09), IEEE Press, pp 156–161

  30. Wikipedia. Super Mario series. http://en.wikipedia.org/wiki/Super_Mario_(series)#Games

Download references

Acknowledgments

This work has been supported in part by the P08-TIC-03903 project awarded by the Andalusian Regional Government, the FPU Grant 2009-2942, the TIN2011-28627-C04-02 project, awarded by the Spanish Ministry of Science and Innovation, and project #83 from CEI-BioTIC at the University of Granada.

Author information

Authors and Affiliations

  1. Departamento de Arquitectura y Tecnología de Computadores, ETSIIT-CITIC, University of Granada, Granada, Spain

    A. M. Mora, J. J. Merelo, P. García-Sánchez, P. A. Castillo, M. S. Rodríguez-Domingo & R. M. Hidalgo-Bermúdez

Authors
  1. A. M. Mora
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. J. J. Merelo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. P. García-Sánchez
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. P. A. Castillo
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. M. S. Rodríguez-Domingo
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. R. M. Hidalgo-Bermúdez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to A. M. Mora.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mora, A.M., Merelo, J.J., García-Sánchez, P. et al. Creating autonomous agents for playing Super Mario Bros game by means of evolutionary finite state machines. Evol. Intel. 6, 205–218 (2014). https://doi.org/10.1007/s12065-014-0105-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-014-0105-7

Keywords

Search

Navigation