Skip to main content
SpringerLink
Log in

Analysis of the Protocols Used to Assess Virtual Players in Multi-player Computer Games

  • Conference paper
  • First Online:
Advances in Computational Intelligence (IWANN 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10306))

Included in the following conference series:

Abstract

Recently, the development of believable agents has gained a lot of interest and many solutions have been proposed by the research community to implement such bots. However, in order to make advances in this field, a generic and rigorous evaluation that would allow the comparison of new systems against existing ones is needed. This paper provides a summary of the existing believability assessments. Seven features characterising the protocols are identified. After a comprehensive analysis, recommendations and prospects for improvement are provided.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Human-Like Bots Competition, presented at the IEEE CIG conference by Raúl Arrabales: http://www.slideshare.net/array2001/arrabales-bot-prize2014v2.

  2. 2.

    http://www.loebner.net/Prizef/loebner-prize.html.

References

  1. Livingstone, D.: Turing’s test and believable AI in games. Comput. Entertainment 4(1), 6 (2006)

    Article  Google Scholar 

  2. Soni, B., Hingston, P.: Bots trained to play like a human are more fun. In: IEEE International Joint Conference on Neural Networks (IEEE World Congress on Computational Intelligence), pp. 363–369, June 2008

    Google Scholar 

  3. van Hoorn, N., Togelius, J., Wierstra, D., Schmidhuber, J.: Robust player imitation using multiobjective evolution. In: 2009 IEEE Congress on Evolutionary Computation, pp. 652–659. IEEE, May 2009

    Google Scholar 

  4. Asensio Llargues, J.M., Peralta, J., Arrabales, R., Bedia, M.G., Cortez, P., Peña, A.L.: Artificial Intelligence approaches for the generation and assessment of believable human-like behaviour in virtual characters. Expert Syst. Appl. 41(16), 7281–7290 (2014)

    Article  Google Scholar 

  5. Laird, J.E., Duchi, J.C.: Creating human-like synthetic characters with multiple skill levels: a case study using the Soar Quakebot. Papers from 2001 AAAI Spring Symposium, Artificial Intelligence and Interactive Entertainment I, pp. 54–58 (2001)

    Google Scholar 

  6. Polceanu, M.: Mirrorbot: using human-inspired mirroring behavior to pass a turing test. In: IEEE Conference on Computational Intelligence in Games (CIG 2013), pp. 1–8. IEEE (2013)

    Google Scholar 

  7. Le Hy, R., Arrigoni, A., Bessière, P., Lebeltel, O.: Teaching Bayesian behaviours to video game characters. Robot. Auton. Syst. 47(2–3), 177–185 (2004)

    Google Scholar 

  8. Gorman, B., Thurau, C., Bauckhage, C., Humphrys, M.: Believability testing and bayesian imitation in interactive computer games. From Anim. Animats 9(1), 655–666 (2006)

    Article  Google Scholar 

  9. Tencé, F., Gaubert, L., Soler, J., Loor, P., Buche, C.: CHAMELEON: online learning for believable behaviors based on humans imitation in computer games. Comput. Animat. Virtual Worlds (CAVW) 24(5), 477–496 (2013)

    Google Scholar 

  10. Mac Namee, B.: Proactive Persistent Agents: Using Situational Intelligence to Create Support Characters in Character-Centric Computer Games. Ph.D thesis, University of Dublin, Trinity College, August 2004

    Google Scholar 

  11. McGlinchey, S., Livingstone, D.: What believability testing can tell us. In: Proceedings of the International Conference on Computer Games: Artificial Intelligence, Design, and Education, pp. 273–277 (2004)

    Google Scholar 

  12. Lucas, S.M., Mateas, M., Preuss, M., Spronck, P., Togelius, J.: Artificial and computational intelligence in games (Dagstuhl Seminar 12191). Dagstuhl Rep. 2(5), 43–70 (2012)

    Google Scholar 

  13. Hinkkanen, T., Kurhila, J., Pasanen, T.A.: Framework for evaluating believability of non-player characters in games. In: AI and Machine Consciousness (2008)

    Google Scholar 

  14. Arrabales, R., Ledezma, A., Sanchis, A.: ConsScale: a pragmatic scale for measuring the level of consciousness in artificial agents. J. Conscious. Stud. 17(3–1), 131–164 (2010)

    Google Scholar 

  15. Arrabales, R., Ledezma, A., Sanchis, A.: ConsScale FPS: cognitive integration for improved believability in computer game bots. In: Hingston, P. (ed.) Believable Bots, pp. 193–214. Springer, Heidelberg (2012)

    Google Scholar 

  16. Togelius, J.: How to run a successful game-based AI competition. IEEE Trans. Comput. Intell. AI Games 8(1), 95–100 (2016)

    Article  Google Scholar 

  17. Hingston, P.: A turing test for computer game bots. IEEE Trans. Comput. Intell. AI Games 1(3), 169–186 (2009)

    Article  Google Scholar 

  18. Shaker, N., Togelius, J., Yannakakis, G.N., Poovanna, L., Ethiraj, V.S., Johansson, S.J., Reynolds, R.G., Heether, L.K., Schumann, T., Gallagher, M.: The turing test track of the 2012 Mario AI Championship: entries and evaluation. In: IEEE Conference on Computational Intelligence in Games (CIG 2013), pp. 1–8. IEEE (2013)

    Google Scholar 

  19. Turing, A.M.: Computing machinery and intelligence. Mind 59(236), 433–460 (1950)

    Article  MathSciNet  Google Scholar 

  20. Hingston, P.: A new design for a Turing Test for Bots. In: Proceedings of the 2010 IEEE Conference on Computational Intelligence and Games, pp. 345–350. IEEE, August 2010

    Google Scholar 

  21. Thawonmas, R., Murakami, S., Sato, T.: Believable judge bot that learns to select tactics and judge opponents. In: IEEE Conference on Computational Intelligence and Games (CIG 2011), pp. 345–349 (2011)

    Google Scholar 

  22. Acampora, G., Loia, V., Vitiello, A.: Improving game bot behaviours through timed emotional intelligence. Knowl.-Based Syst. 34, 97–113 (2012)

    Article  Google Scholar 

  23. Tencé, F., Buche, C., De Loor, P., Marc, O.: The challenge of believability in video games: definitions, agents models and imitation learning. In Mao, W., Vermeersch, L. (eds.) 2nd Asian Conference on Simulation and AI in Computer Games (GAMEON-ASIA 2010), Eurosis, pp. 38–45 (2010)

    Google Scholar 

  24. Togelius, J., Yannakakis, G.N., Karakovskiy, S., Shaker, N.: Assessing believability. In: Hingston, P. (ed.) Believable Bots: Can Computers Play Like People?, pp. 215–230. Springer, Heidelberg (2012)

    Google Scholar 

  25. Bossard, C., Benard, R., De Loor, P., Kermarrec, G., Tisseau, J.: An exploratory evaluation of virtual football player’s believability. In: Proceedings of 11th Virtual Reality International Conference (VRIC 2009), pp. 171–172 (2009)

    Google Scholar 

  26. Bogdanovych, A., Trescak, T., Simoff, S.: What makes virtual agents believable? Connection Sci. 28(1), 83–108 (2016)

    Article  Google Scholar 

  27. Bernacchia, M., Hoshino, J.: AI platform for supporting believable combat in role-playing games. In: Proceedings of the 19th Game Programming Workshop in Japan, pp. 139–144 (2014)

    Google Scholar 

  28. Cheung, G., Huang, J.: Starcraft from the stands: understanding the game spectator. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 763–772 (2011)

    Google Scholar 

  29. Kaytoue, M., Silva, A., Cerf, L.: Watch me playing, i am a professional: a first study on video game live streaming. In: Proceedings of the 21st International Conference Companion on World Wide Web, June 2009, pp. 1181–1188 (2012)

    Google Scholar 

  30. Paritosh, P., Marcus, G.: Toward a comprehension challenge, using crowdsourcing as a tool. AI Mag. 37(1), 23–30 (2016)

    Google Scholar 

  31. Brace, I.: Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research. Kogan Page Publishers (2008)

    Google Scholar 

  32. Hyman, H.H., National Opinion Research Center: Interviewing in social research. A research project of the National Opinion Research Center. University of Chicago Press, Chicago (1954)

    Google Scholar 

  33. Weibel, D., Wissmath, B., Habegger, S., Steiner, Y., Groner, R.: Playing online games against computer- vs. human-controlled opponents: effects on presence, flow, and enjoyment. Comput. Hum. Behav. 24(5), 2274–2291 (2008)

    Article  Google Scholar 

  34. Lim, S., Reeves, B.: Computer agents versus avatars: responses to interactive game characters controlled by a computer or other player. Int. J. Hum. Comput. Stud. 68(1–2), 57–68 (2010)

    Article  Google Scholar 

  35. Friedman, H.H., Amoo, T.: Rating the rating scales. J. Market. Manag. 9(3), 114–123 (1999)

    Google Scholar 

  36. Yannakakis, G.N., Hallam, J.: Real-time game adaptation for optimizing player satisfaction. IEEE Trans. Comput. Intell. AI Games 1(2), 121–133 (2009)

    Article  Google Scholar 

  37. Clark, P., Etzioni, O.: My computer is an honor student but how intelligent is it? standardized tests as a measure of AI. AI Mag. 37(1), 5–12 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Lab-STICC, ENIB, CERV, 25 Rue Claude Chappe, 29280, Plouzané, France

    Cindy Even, Anne-Gwenn Bosser & Cédric Buche

  2. Virtualys, 41 Rue Yves Collet, 29200, Brest, France

    Cindy Even

Authors
  1. Cindy Even
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Anne-Gwenn Bosser
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cédric Buche
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cindy Even .

Editor information

Editors and Affiliations

  1. Universidad de Granada, Granada, Spain

    Ignacio Rojas

  2. University of Malaga, Malaga, Spain

    Gonzalo Joya

  3. Polytechnic University of Catalonia, Vilanova i la Geltrú, Barcelona, Spain

    Andreu Catala

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Even, C., Bosser, AG., Buche, C. (2017). Analysis of the Protocols Used to Assess Virtual Players in Multi-player Computer Games. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2017. Lecture Notes in Computer Science(), vol 10306. Springer, Cham. https://doi.org/10.1007/978-3-319-59147-6_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-59147-6_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-59146-9

  • Online ISBN: 978-3-319-59147-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation