Skip to main content
SpringerLink
Log in

An Industrial Experience Leveraging the iv4XR Framework for BDD Testing of a 3D Sandbox Game

  • Conference paper
  • First Online:
Research Challenges in Information Science (RCIS 2024)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 513))

Included in the following conference series:

  • 26 Accesses

Abstract

Industrial-grade games, like Space Engineers, must adopt swift development and testing processes to conform to rigorous quality standards. Nevertheless, the testing phase of these extensive and complex games heavily relies on manual effort from play-testers, leading to productivity constraints during development cycles. This experience paper reports a Behavior-Driven-Development (BDD) software development process for automated regression test scenarios that allows complement testers’ work during development cycles. To enable BDD test scripts for the Space Engineers game, we have extended the iv4XR framework into a game plugin to connect and execute game actions. Additionally, we have integrated the Cucumber software to describe game test scenarios using natural language. This approach allows testers to create, maintain, and execute a subset of regression test scenarios by relying on a BDD agent that can autonomously verify Space Engineers game features, enabling seamless integration into the development cycle.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.atlassian.com/software/jira.

  2. 2.

    https://www.gurock.com/testrail/.

  3. 3.

    https://github.com/iv4xr-project/iv4xr-se-plugin.

References

  1. de Andrade, S.A., Nunes, F.L., Delamaro, M.E.: Exploiting deep reinforcement learning and metamorphic testing to automatically test virtual reality applications. STVR 33(8), e1863 (2023)

    Google Scholar 

  2. Ariyurek, S., Surer, E., Betin-Can, A.: Playtesting: what is beyond personas. IEEE Trans. Games (2022)

    Google Scholar 

  3. Bahaweres, R.B., et al.: Behavior-driven development (BDD) cucumber katalon for automation GUI testing case CURA and swag labs. In: ICIMCIS, pp. 87–92 (2020)

    Google Scholar 

  4. Binamungu, L.P., Maro, S.: Behaviour driven development: a systematic mapping study. J. Syst. Softw. 111749 (2023)

    Google Scholar 

  5. Bucchiarone, A., Cooper, K.M., Lin, D., Melcer, E.F., Sung, K.: Games and software engineering: engineering fun, inspiration, and motivation. ACM SIGSOFT Softw. Eng. Notes 48(1), 85–89 (2023)

    Article  Google Scholar 

  6. Coppola, R., Morisio, M., Torchiano, M.: Mobile GUI testing fragility: a study on open-source android applications. Trans. Reliab. 68(1), 67–90 (2018)

    Article  Google Scholar 

  7. Dees, I., Wynne, M., Hellesoy, A.: Cucumber Recipes: Automate Anything with BDD Tools and Techniques. Pragmatic Bookshelf (2013)

    Google Scholar 

  8. García, B., Gallego, M., Gortázar, F., Munoz-Organero, M.: A survey of the selenium ecosystem. Electronics 9(7), 1067 (2020)

    Article  Google Scholar 

  9. Gordillo, C., Bergdahl, J., Tollmar, K., Gisslén, L.: Improving playtesting coverage via curiosity driven reinforcement learning agents. In: 2021 IEEE Conference on Games (CoG), pp. 1–8. IEEE (2021)

    Google Scholar 

  10. Hammoudi, M., Rothermel, G., Tonella, P.: Why do record/replay tests of web applications break? In: International Conference ICST, pp. 180–190. IEEE (2016)

    Google Scholar 

  11. Nguyen, V., To, T., Diep, G.H.: Generating and selecting resilient and maintainable locators for web automated testing. STVR 31(3), e1760 (2021)

    Google Scholar 

  12. Nie, L., Said, K.S., Ma, L., Zheng, Y., Zhao, Y.: A systematic mapping study for graphical user interface testing on mobile apps. IET Software 17(3), 249–267 (2023)

    Article  Google Scholar 

  13. North, D., et al.: Introducing BDD. Better Softw. 12, 7 (2006)

    Google Scholar 

  14. Paduraru, C., Paduraru, M., Stefanescu, A.: Rivergame-a game testing tool using artificial intelligence. In: International Conference ICST, pp. 422–432. IEEE (2022)

    Google Scholar 

  15. Pereira, L., Sharp, H., de Souza, C., Oliveira, G., Marczak, S., Bastos, R.: Behavior-driven development benefits and challenges: reports from an industrial study. In: ICASD Companion, pp. 1–4 (2018)

    Google Scholar 

  16. Pezze, M., Rondena, P., Zuddas, D.: Automatic GUI testing of desktop applications: an empirical assessment of the state of the art. In: ISSTA, pp. 54–62 (2018)

    Google Scholar 

  17. Pfau, J., Smeddinck, J.D., Malaka, R.: Automated game testing with ICARUS: intelligent completion of adventure riddles via unsupervised solving. In: CHI PLAY: Extended Abstracts, pp. 153–164. ACM (2017)

    Google Scholar 

  18. Politowski, C., Guéhéneuc, Y.G., Petrillo, F.: Towards automated video game testing: still a long way to go. In: 2022 IEEE/ACM 6th International Workshop on Games and Software Engineering (GAS), pp. 37–43. IEEE (2022)

    Google Scholar 

  19. Politowski, C., Petrillo, F., Guéhéneuc, Y.G.: A survey of video game testing. In: International Conference on Automation of Software Test (AST), pp. 90–99. IEEE (2021)

    Google Scholar 

  20. Prasetya, I., Dastani, M., Prada, R., Vos, T., Dignum, F., Kifetew, F.: Aplib: tactical agents for testing computer games. In: 8th EMAS Workshop (2020)

    Google Scholar 

  21. Prasetya, I., et al.: An agent-based approach to automated game testing: an experience report. In: A-TEST, pp. 1–8 (2022)

    Google Scholar 

  22. Ralph, P., Tempero, E.: Construct validity in software engineering research and software metrics. In: 22nd International Conference EASE, pp. 13–23 (2018)

    Google Scholar 

  23. Rodríguez-Valdés, O., Vos, T., Aho, P., Marín, B.: 30 years of automated GUI testing: a bibliometric analysis. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds.) QUATIC 2021, pp. 473–488. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85347-1_34

  24. Sestini, A., Gisslén, L., Bergdahl, J., Tollmar, K., Bagdanov, A.D.: Automated gameplay testing and validation with curiosity-conditioned proximal trajectories. IEEE Trans. Games 16(1), 113–126 (2022)

    Article  Google Scholar 

  25. Smart, J., Molak, J.: BDD in Action. Simon and Schuster (2023)

    Google Scholar 

  26. Walker, A.M.: Tacit knowledge. Eur. J. Epidemiol. 32(4), 261–267 (2017)

    Article  Google Scholar 

  27. Wang, X.: Vrtest: an extensible framework for automatic testing of virtual reality scenes. In: 2022 IEEE/ACM 44th ICSE-Companion, pp. 232–236. IEEE (2022)

    Google Scholar 

  28. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-29044-2

  29. Wu, Y., Chen, Y., Xie, X., Yu, B., Fan, C., Ma, L.: Regression testing of massively multiplayer online role-playing games. In: 2020 IEEE ICSME, pp. 692–696. IEEE (2020)

    Google Scholar 

  30. Zheng, Y., et al.: Wuji: automatic online combat game testing using evolutionary deep reinforcement learning. In: 34th International Conference ASE, pp. 772–784. IEEE (2019)

    Google Scholar 

Download references

Acknowledgment

This work has been partially funded by: H2020 EU iv4XR grant nr. 856716 and ENACTEST ERASMUS+ grant nr. 101055874.

Author information

Authors and Affiliations

  1. Universitat Politècnica de València, València, Spain

    Fernando Pastor Ricós, Beatriz Marín & Tanja E. J. Vos

  2. Utrecht University, Utrecht, The Netherlands

    I. S. W. B. Prasetya

  3. Open Universiteit, Heerlen, The Netherlands

    Tanja E. J. Vos

  4. GoodAI, Prague, Czechia

    Joseph Davidson & Karel Hovorka

Authors
  1. Fernando Pastor Ricós
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beatriz Marín
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. I. S. W. B. Prasetya
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tanja E. J. Vos
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Joseph Davidson
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karel Hovorka
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fernando Pastor Ricós .

Editor information

Editors and Affiliations

  1. NOVA University Lisbon, Caparica, Portugal

    João Araújo

  2. University of Castilla La Mancha, Albacete, Albacete, Spain

    Jose Luis de la Vara

  3. University of Minho, Guimarães, Portugal

    Maribel Yasmina Santos

  4. Institut Mines-Télécom Business School, Evry, France

    Saïd Assar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pastor Ricós, F., Marín, B., Prasetya, I.S.W.B., Vos, T.E.J., Davidson, J., Hovorka, K. (2024). An Industrial Experience Leveraging the iv4XR Framework for BDD Testing of a 3D Sandbox Game. In: Araújo, J., de la Vara, J.L., Santos, M.Y., Assar, S. (eds) Research Challenges in Information Science. RCIS 2024. Lecture Notes in Business Information Processing, vol 513. Springer, Cham. https://doi.org/10.1007/978-3-031-59465-6_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-59465-6_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-59464-9

  • Online ISBN: 978-3-031-59465-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation