Skip to main content
SpringerLink
Log in

A Framework for the Gamification of GUI Testing

  • Chapter
  • First Online:
Software Engineering for Games in Serious Contexts

Abstract

Software testing is a critical activity in the software development process. Several techniques have been proposed, addressing different levels of granularity from low-level unit testing to higher-level exploratory testing through the software’s graphical user interface (GUI). In modern software development, most test cases are obtained by automated test generation. However, while automation generally achieves high coverage in code-level white-box testing, it does not always generate realistic sequences of interactions with the GUI. By contrast, manual exploratory testing has survived as a costly, error-prone, and tedious yet crucial activity. Gamification is seen as an opportunity to increase user satisfaction and engagement while performing testing activities. It could also enable and encourage crowdsourced testing tasks. The purpose of the study described in this chapter is to provide a framework of gamification mechanics and dynamics that can be applied to the practice of manual exploratory GUI testing. We provide an implementation of the framework as an extension of an existing manual exploratory GUI testing for Web applications, and we provide a preliminary evaluation of the gamified tool in terms of provided efficiency, effectiveness, and user experience. Our results show that the gamified solution makes the testers obtain test suites with higher coverage while reducing slightly the number of bugs signalled while traversing the applications under test. The gamified tool also was considered to provide a positive user experience, and the majority of participants expressed their willingness to use such instruments again in the future. As future work, we foresee the implementation of the framework in a stand-alone tool and in-depth empirical experiment to evaluate quantitatively the benefits and drawbacks provided by such mechanics in real testing scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 179.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Borjesson, E., Feldt, R.: Automated system testing using visual GUI testing tools: a comparative study in industry. In: 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation, pp. 350–359. IEEE, Piscataway (2012)

    Google Scholar 

  2. Pedreira, O., García, F., Brisaboa, N., Piattini, M.: Gamification in software engineering–a systematic mapping. Inf. Softw. Technol. 57, 157–168 (2015)

    Article  Google Scholar 

  3. Mäntylä, M.V., Smolander, K.: Gamification of software testing-an MLR. In: International Conference on Product-Focused Software Process Improvement, pp. 611–614. Springer, Berlin (2016)

    Google Scholar 

  4. Rodrigues, L., Pereira, F.D., Toda, A.M., Palomino, P.T., Pessoa, M., Carvalho, L.S.G., Fernandes, D., Oliveira, E.H., Cristea, A.I., Isotani, S.: Gamification suffers from the novelty effect but benefits from the familiarization effect: findings from a longitudinal study. Int. J. Educ. Technol. High. Educ. 19(1), 1–25 (2022)

    Article  Google Scholar 

  5. Chou, Y.-k.: Actionable Gamification: Beyond Points, Badges, and Leaderboards. Packt Publishing Ltd, Birmingham (2019)

    Google Scholar 

  6. Hosseini, C., Humlung, O., Fagerstrøm, A., Haddara, M.: An experimental study on the effects of gamification on task performance. In: Procedia Computer Science 196 (2022) 999–1006, International Conference on ENTERprise Information Systems/ProjMAN – International Conference on Project MANagement/HCist – International Conference on Health and Social Care Information Systems and Technologies 2021. https://www.sciencedirect.com/science/article/pii/S1877050921023255

  7. Wang, C., He, J., Jin, Z., Pan, S., Lafkihi, M., Kong, X.: The impact of gamification on teaching and learning physical internet: a quasi-experimental study. Ind. Manag. Data Syst. 122, 1499–1521 (2022)

    Article  Google Scholar 

  8. Jensen, M.L., Wright, R.T., Durcikova, A., Karumbaiah, S.: Improving phishing reporting using security gamification. J. Manag. Inf. Syst. 39(3), 793–823 (2022)

    Article  Google Scholar 

  9. Liechti, O., Pasquier, J., Reis, R.: Supporting agile teams with a test analytics platform: a case study. In: 2017 IEEE/ACM 12th International Workshop on Automation of Software Testing (AST), pp. 9–15. IEEE, Piscataway (2017)

    Google Scholar 

  10. Cacciotto, F., Fulcini, T., Coppola, R., Ardito, L.: A metric framework for the gamification of web and mobile GUI testing. In: 2021 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 126–129 (2021)

    Google Scholar 

  11. Fulcini, T., Ardito, L.: Gamified exploratory GUI testing of web applications: a preliminary evaluation. In: 2022 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 215–222 (2022)

    Google Scholar 

  12. Alégroth, E., Gao, Z., Oliveira, R., Memon, A.: Conceptualization and evaluation of component-based testing unified with visual GUI testing: an empirical study. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10. IEEE, Piscataway (2015)

    Google Scholar 

  13. Ardito, L., Bottino, A., Coppola, R., Lamberti, F., Manigrasso, F., Morra, L., Torchiano, M.: Feature matching-based approaches to improve the robustness of android visual GUI testing. ACM Trans. Softw. Eng. Methodol. 31(2), 1–32 (2021)

    Article  Google Scholar 

  14. Vasquez, M.L., Moran, K., Poshyvanyk, D.: Continuous, evolutionary and large-scale: a new perspective for automated mobile app testing. Preprint. arXiv:1801.06267

    Google Scholar 

  15. Coppola, R., Morisio, M., Torchiano, M.: Mobile GUI testing fragility: a study on open-source android applications. IEEE Trans. Reliab. 68(1), 67–90 (2018)

    Article  Google Scholar 

  16. Memon, A.M., Cohen, M.B.: Automated testing of GUI applications: models, tools, and controlling flakiness. In: 2013 35th International Conference on Software Engineering (ICSE), pp. 1479–1480. IEEE, Piscataway (2013)

    Google Scholar 

  17. Kamran, M., Rashid, J., Nisar, M.W.: Android fragmentation classification, causes, problems and solutions. Int. J. Comput. Sci. Inf. Sec. 14(9), 992 (2016)

    Google Scholar 

  18. Nass, M., Alégroth, E., Feldt, R.: Why many challenges with GUI test automation (will) remain. Inf. Softw. Technol. 138, 106625 (2021)

    Article  Google Scholar 

  19. Kochhar, P.S., Thung, F., Nagappan, N., Zimmermann, T., Lo, D.: Understanding the test automation culture of app developers. In: 2015 IEEE 8th International Conference on Software Testing, Verification and Validation (ICST), pp. 1–10. IEEE, Piscataway (2015)

    Google Scholar 

  20. Krutz, D.E., Malachowsky, S.A., Reichlmayr, T.: Using a real world project in a software testing course. In: Proceedings of the 45th ACM Technical Symposium on Computer Science Education, pp. 49–54 (2014)

    Google Scholar 

  21. Barreto, C.F., França, C., Gamification in software engineering: a literature review. In: 2021 IEEE/ACM 13th International Workshop on Cooperative and Human Aspects of Software Engineering (CHASE), pp. 105–108. IEEE, Piscataway (2021)

    Google Scholar 

  22. de Melo, A.A., Hinz, M., Scheibel, G., Berkenbrock, C.D.M., Gasparini, I., Baldo, F.: Version control system gamification: a proposal to encourage the engagement of developers to collaborate in software projects. In: Meiselwitz, G. (ed.) Social Computing and Social Media, pp. 550–558. Springer International Publishing, New York City (2014)

    Chapter  Google Scholar 

  23. Ašeriškis, D., Damaševičius, R.: Gamification of a project management system. In: The Seventh International Conference on Advances in Computer-Human Interactions, pp. 200–207 (2014)

    Google Scholar 

  24. Parizi, R.M.: On the gamification of human-centric traceability tasks in software testing and coding. In: 2016 IEEE 14th International Conference on Software Engineering Research, Management and Applications (SERA), pp. 193–200 (2016)

    Google Scholar 

  25. Ruohonen, J., Allodi, L.: A bug bounty perspective on the disclosure of web vulnerabilities. ArXiv abs/1805.09850

    Google Scholar 

  26. Üsfekes, Ç., Tüzün, E., Yilmaz, M., Macit, Y., Clarke, P.M.: Auction-based serious game for bug tracking. IET Softw. 13, 386–392 (2019)

    Article  Google Scholar 

  27. Bell, J., Sheth, S., Kaiser, G.: Secret ninja testing with halo software engineering. In: Proceedings of the 4th International Workshop on Social Software Engineering, SSE ’11, pp. 43–47. Association for Computing Machinery, New York (2011)

    Google Scholar 

  28. Silvis-Cividjian, N., Limburg, R., Althuisius, N., Apostolov, E., Bonev, V., Jansma, R., Visser, G., Went, M.: Vu-bugzoo: a persuasive platform for teaching software testing. In: Proceedings of the 2020 ACM Conference on Innovation and Technology in Computer Science Education, ITiCSE ’20, p. 553. Association for Computing Machinery, New York (2020)

    Google Scholar 

  29. P.E., Y.F., Clarke, P.J.: Gamification-based cyber-enabled learning environment of software testing. In: 2016 ASEE Annual Conference & Exposition, ASEE Conferences, New Orleans, Louisiana, pp. 1–16 (2016). https://peer.asee.org/27000

  30. Clegg, B.S., Rojas, J.M., Fraser, G.: Teaching software testing concepts using a mutation testing game. In: 2017 IEEE/ACM 39th International Conference on Software Engineering: Software Engineering Education and Training Track (ICSE-SEET), pp. 33–36 (2017)

    Google Scholar 

  31. Sun, Y.: Design and implementation of a gamified training system for software testing. Adv. Educ. 10, 395–400 (2020)

    Article  Google Scholar 

  32. Fraser, G., Gambi, A., Kreis, M., Rojas, J.M.: Gamifying a software testing course with code defenders. In: Proceedings of the 50th ACM Technical Symposium on Computer Science Education, SIGCSE ’19, pp. 571–577. Association for Computing Machinery, New York (2019)

    Google Scholar 

  33. I. 9001:2005, Quality management systems – requirements, Standard, International Organization for Standardization (2005)

    Google Scholar 

  34. Lőrincz, B., Iudean, B., Vescan, A.: Experience report on teaching testing through gamification. In: EASEAI 2021, pp. 15–22. Association for Computing Machinery, New York (2021)

    Google Scholar 

  35. Amiri-Chimeh, S., Haghighi, H., Vahidi-Asl, M., Setayesh-Ghajar, K., Gholami-Ghavamabad, F.: Rings: a game with a purpose for test data generation. Interact. Comput. 30(1), 1–30 (2017)

    Article  Google Scholar 

  36. Fraser, G., Gambi, A., Rojas, J.M.: Teaching software testing with the code defenders testing game: experiences and improvements. In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 461–464 (2020)

    Google Scholar 

  37. Bryce, R., Mayo, Q., Andrews, A., Bokser, D., Burton, M., Day, C., Gonzolez, J., Noble, T.: Bug catcher: a system for software testing competitions. In: Proceeding of the 44th ACM Technical Symposium on Computer Science Education, SIGCSE ’13, pp. 513–518. Association for Computing Machinery, New York (2013)

    Google Scholar 

  38. García, F., Pedreira, O., Piattini, M., Cerdeira-Pena, A., Penabad, M.: A framework for gamification in software engineering. J. Syst. Softw. 132, 21–40 (2017). https://www.sciencedirect.com/science/article/pii/S0164121217301218

  39. Berkling, K., Thomas, C.: Gamification of a software engineering course and a detailed analysis of the factors that lead to it’s failure. In: 2013 International Conference on Interactive Collaborative Learning (ICL), pp. 525–530 (2013)

    Google Scholar 

  40. Herranz, E., Colomo-Palacios, R., Al-Barakati, A.: Deploying a gamification framework for software process improvement: preliminary results. In: Stolfa, J., Stolfa, S., O’Connor, R.V., Messnarz, R. (eds.) Systems, Software and Services Process Improvement, pp. 231–240. Springer International Publishing, Cham (2017)

    Chapter  Google Scholar 

  41. de Jesus, G.M., Paschoal, L.N., Ferrari, F.C., Souza, S.R.S.: Is it worth using gamification on software testing education? An experience report. In: Proceedings of the XVIII Brazilian Symposium on Software Quality, SBQS’19, pp. 178–187. Association for Computing Machinery, New York (2019)

    Google Scholar 

  42. Pedreira, O., García, F., Piattini, M., Cortiñas, A., Cerdeira-Pena, A.: An architecture for software engineering gamification. Tsinghua Sci. Technol. 25(6), 776–797 (2020)

    Article  Google Scholar 

  43. Jesus, G.M.d., Ferrari, F.C., Paschoal, L.N., Souza, S.d.R.S.d., Porto, D.d.P., Durelli, V.H.S.: Is it worth using gamification on software testing education? An extended experience report in the context of undergraduate students. J. Softw. Eng. Res. Dev. 8, 6:1–6:19 (2020). https://sol.sbc.org.br/journals/index.php/jserd/article/view/738

  44. Nass, M., Alégroth, E., Feldt, R.: Augmented testing: industry feedback to shape a new testing technology. In: 2019 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 176–183. IEEE, Piscataway (2019)

    Google Scholar 

  45. Lee, Y., Kozar, K.A., Larsen, K.R.: The technology acceptance model: past, present, and future. Commun. Assoc. Inf. Syst. 12(1), 50 (2003)

    Google Scholar 

  46. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer Science & Business Media, Berlin (2012)

    Book  MATH  Google Scholar 

  47. Coppola, R., Alégroth, E.: A taxonomy of metrics for gui-based testing research: a systematic literature review. Inf. Softw. Technol. 152, 107062 (2022). https://www.sciencedirect.com/science/article/pii/S0950584922001719

Download references

Author information

Authors and Affiliations

  1. Politecnico di Torino, Department of Control and Computer Engineering, Torino, Italy

    Riccardo Coppola, Luca Ardito, Tommaso Fulcini, Giacomo Garaccione, Marco Torchiano & Maurizio Morisio

Authors
  1. Riccardo Coppola
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Luca Ardito
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tommaso Fulcini
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Giacomo Garaccione
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Marco Torchiano
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Maurizio Morisio
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Riccardo Coppola .

Editor information

Editors and Affiliations

  1. Kelowna, BC, Canada

    Kendra M. L. Cooper

  2. Fondazione Bruno Kessler, Trento, Italy

    Antonio Bucchiarone

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Coppola, R., Ardito, L., Fulcini, T., Garaccione, G., Torchiano, M., Morisio, M. (2023). A Framework for the Gamification of GUI Testing. In: Cooper, K.M.L., Bucchiarone, A. (eds) Software Engineering for Games in Serious Contexts. Springer, Cham. https://doi.org/10.1007/978-3-031-33338-5_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-33338-5_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-33337-8

  • Online ISBN: 978-3-031-33338-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation