Skip to main content
SpringerLink
Log in

Novelty-Driven Evolutionary Scriptless Testing

  • Conference paper
  • First Online:
Research Challenges in Information Science (RCIS 2024)

Part of the book series: Lecture Notes in Business Information Processing ((LNBIP,volume 514))

Included in the following conference series:

  • 41 Accesses

Abstract

In recent years, scriptless Graphical User Interface (gui) testing has been positioned as a complement to traditional testing techniques. Automated scriptless GUI testing approaches use Action Selection Rules (asr) to generate on-the-fly test sequences when testing a software system. Currently, random is the standard selection approach in scriptless testing, provoking drawbacks in the testing process, such as test sequences that do not reflect the human strategies for testing, and being unable to deal with multistep tasks. This paper presents an alternate selection approach based on the use of a grammar to design the asr and an Evolutionary Algorithm (ea) with Novelty Search (ns) to direct the evolution process. Preliminary testing shows that the asrs do evolve in the standard ea process. Further research is needed to show the benefits of the additional ns for the testing process.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 99.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Amalfitano, D., Amatucci, N., Fasolino, A.R., Tramontana, P.: Agrippin: a novel search based testing technique for android applications. In: 3rd International Workshop on Software Development Lifecycle for Mobile, pp. 5-12. DeMobile 2015, ACM (2015)

    Google Scholar 

  2. Bauersfeld, S., Vos, T.E., Condori-Fernández, N., Bagnato, A., Brosse, E.: Evaluating the testar tool in an industrial case study. In: 8th ACM/IEEE International Symposium on Empirical Software Engineering and Measurement, pp. 1–9 (2014)

    Google Scholar 

  3. Bertolino, A.: Software testing research: achievements, challenges, dreams. In: Future of Software Engineering (FOSE’07), pp. 85–103. IEEE (2007)

    Google Scholar 

  4. Beyer, H.G., Schwefel, H.P.: Evolution strategies - a comprehensive introduction. Nat. Comput. 1(1), 3–52 (2002)

    Article  MathSciNet  Google Scholar 

  5. Bons, A., Marín, B., Aho, P., Vos, T.E.: Scripted and scriptless gui testing for web applications: an industrial case. Inf. Softw. Technol. 158, 107172 (2023)

    Article  Google Scholar 

  6. Chahim, H., Duran, M., Vos, T.E.J., Aho, P., Condori Fernandez, N.: Scriptless Testing at the GUI Level in an Industrial Setting. In: Dalpiaz, F., Zdravkovic, J., Loucopoulos, P. (eds.) Research Challenges in Information Science: 14th International Conference, RCIS 2020, Limassol, Cyprus, September 23–25, 2020, Proceedings, pp. 267–284. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-50316-1_16

    Chapter  Google Scholar 

  7. DeVries, B., Trefftz, C.: A novelty search and metamorphic testing approach to automatic test generation. In: 2021 IEEE/ACM 14th International Workshop on Search-Based Software Testing (SBST), pp. 8–11 (2021)

    Google Scholar 

  8. Dorigo, M., Blum, C.: Ant colony optimization theory: a survey. Theoret. Comput. Sci. 344(2), 243–278 (2005)

    Article  MathSciNet  Google Scholar 

  9. Esparcia-Alcázar, A., Almenar, F., Vos, T.E.J., Rueda, U.: Using genetic programming to evolve action selection rules in traversal-based automated software testing: results obtained with the testar tool. Memetic Comput. 10(3), 257–265 (2018)

    Article  Google Scholar 

  10. Fenton, M., McDermott, J., Fagan, D., Forstenlechner, S., Hemberg, E., O’Neill, M.: Ponyge2: grammatical evolution in python. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion. GECCO ’17, ACM (Jul 2017)

    Google Scholar 

  11. de Groot, M.: Using evolutionary computing to improve black box monkey testing on a Graphical User Interface. Master’s thesis, Open Universiteit, Heerlen, Netherlands (Apr 2018)

    Google Scholar 

  12. Harman, M., Jones, B.F.: Search-based software engineering. Inform. Softw. Technol.43, 833–839 (2001)

    Google Scholar 

  13. Hufkens, L.V.: Grammar-based action selection rules for scriptless testing. In: To be published in 5th ACM/IEEE International Conference on Automation of Software Test (AST). ACM (2024)

    Google Scholar 

  14. Hufkens, L.V.: Evolutionary scriptless testing. In: Guizzardi, R., Ralyté, J., Franch, X. (eds.) Research Challenges in Information Science: 16th International Conference, RCIS 2022, Barcelona, Spain, May 17–20, 2022, Proceedings, pp. 779–785. Springer International Publishing, Cham (2022). https://doi.org/10.1007/978-3-031-05760-1_55

    Chapter  Google Scholar 

  15. Khari, M., Kumar, P.: An extensive evaluation of search-based software testing: a review. Soft. Comput. 23(6), 1933–1946 (2019)

    Article  Google Scholar 

  16. Lambora, A., Gupta, K., Chopra, K.: Genetic algorithm- a literature review. In: 2019 International Conference on Machine Learning, Big Data, Cloud and Parallel Computing (COMITCon), pp. 380–384 (2019)

    Google Scholar 

  17. Latiu, G.I., Creţ, O., Văcariu, L.: Evoguitest - a graphical user interface testing framework based on evolutionary algorithms. In: 5th International Joint Conference on Computational Intelligence - Volume 1: ECTA. IJCCI 2013, pp. 75–82. SciTePress, INSTICC (2013)

    Google Scholar 

  18. Lehman, J., Stanley, K.O.: Efficiently evolving programs through the search for novelty. In: Proceedings of the 12th Annual Conference on Genetic and Evolutionary Computation, pp. 837–844. ACM, Portland Oregon USA (Jul 2010)

    Google Scholar 

  19. Lehman, J., Stanley, K.O.: Novelty search and the problem with objectives. In: Riolo, R., Vladislavleva, E., Moore, J.H. (eds.) Genetic programming theory and practice IX, pp. 37–56. Springer, New York, New York, NY (2011)

    Chapter  Google Scholar 

  20. McMinn, P.: Search-based software testing: past, present and future. In: 2011 IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, pp. 153–163 (March 2011)

    Google Scholar 

  21. Meinke, K., Walkinshaw, N.: Model-based testing and model inference. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change, pp. 440–443. Springer Berlin Heidelberg, Berlin, Heidelberg (2012). https://doi.org/10.1007/978-3-642-34026-0_32

    Chapter  Google Scholar 

  22. Menninghaus, M., Wilke, F., Schleutker, J.P., Pulvermüller, E.: Search based gui test generation in java - comparing code-based and efg-based optimization goals. In: Proceedings of the 12th International Conference on Evaluation of Novel Approaches to Software Engineering. vol. 1, pp. 179–186. INSTICC, SciTePress (2017)

    Google Scholar 

  23. Papanikolaou, K.: Software testing: a never-ending adventure. https://ginbits.com/software-testing-a-never-ending-adventure/. Accessed 1 Feb 2024

  24. Parasoft: Parabank demo application (2017, 2022). https://github.com/parasoft/parabank/. Accessed 2 Feb 2022

  25. Ricós, F.P., Aho, P., Vos, T., Boigues, I.T., Blasco, E.C., Martínez, H.M.: Deploying TESTAR to enable remote testing in an industrial CI pipeline: a case-based evaluation. In: Margaria, T., Steffen, B. (eds.) Leveraging Applications of Formal Methods, Verification and Validation: Verification Principles: 9th International Symposium on Leveraging Applications of Formal Methods, ISoLA 2020, Rhodes, Greece, October 20–30, 2020, Proceedings, Part I, pp. 543–557. Springer International Publishing, Cham (2020). https://doi.org/10.1007/978-3-030-61362-4_31

    Chapter  Google Scholar 

  26. Ricós, F.P., Neeft, R., Marín, B., Vos, T.E.J., Aho, P.: Using GUI change detection for delta testing. In: Nurcan, S., Opdahl, A.L., Mouratidis, H., Tsohou, A. (eds.) Research Challenges in Information Science: Information Science and the Connected World: 17th International Conference, RCIS 2023, Corfu, Greece, May 23–26, 2023, Proceedings, pp. 509–517. Springer Nature Switzerland, Cham (2023). https://doi.org/10.1007/978-3-031-33080-3_32

    Chapter  Google Scholar 

  27. Pastor Ricós, F., Slomp, A., Marín, B., Aho, P., Vos, T.E.: Distributed state model inference for scriptless GUI testing. J. Syst. Softw. 200, 111645 (2023)

    Article  Google Scholar 

  28. Poli, R., Langdon, W.B., McPhee, N.F., Koza, J.R.: A field guide to genetic programming. Lulu Press], [Morrisville, NC (2008)

    Google Scholar 

  29. Stoyanov, N.: Strategy based genetic algorithms approach in automated GUI testing. Master’s thesis, TU/e, Eindhoven, Netherlands (Sep 2020)

    Google Scholar 

  30. TESTAR: Testar on github. https://github.com/TESTARtool/TESTAR_dev

  31. TESTAR: Testar project download page. https://testar.org/download/

  32. Theuws, G.: Random action selection vs genetic programming: a case study in TESTAR. Master’s thesis, Open Universiteit, Heerlen, Netherlands (Feb 2020)

    Google Scholar 

  33. Villanueva, O.M., Trujillo, L., Hernandez, D.E.: Novelty search for automatic bug repair. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, pp. 1021-1028. GECCO ’20, ACM, New York, NY, USA (2020)

    Google Scholar 

  34. Vos, T.E.J., Aho, P., Ricos, F.P., Rodriguez-Valdes, O., Mulders, A.: Testar - scriptless testing through graphical user interface. Softw. Test. Verif. Reliab. 31(3), e1771 (2021)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Open Universiteit, Valkenburgerweg 177, 6419 AT, Heerlen, The Netherlands

    Lianne V. Hufkens & Tanja E. J. Vos

  2. Universitat Politècnica de València, València, Spain

    Tanja E. J. Vos & Beatriz Marín

Authors
  1. Lianne V. Hufkens
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tanja E. J. Vos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Beatriz Marín
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lianne V. Hufkens .

Editor information

Editors and Affiliations

  1. NOVA University Lisbon, Caparica, Portugal

    João Araújo

  2. University of Castilla La Mancha, Albacete, Albacete, Spain

    Jose Luis de la Vara

  3. University of Minho, Guimarães, Portugal

    Maribel Yasmina Santos

  4. Institut Mines-Télécom Business School, Evry, France

    Saïd Assar

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Hufkens, L.V., Vos, T.E.J., Marín, B. (2024). Novelty-Driven Evolutionary Scriptless Testing. In: Araújo, J., de la Vara, J.L., Santos, M.Y., Assar, S. (eds) Research Challenges in Information Science. RCIS 2024. Lecture Notes in Business Information Processing, vol 514. Springer, Cham. https://doi.org/10.1007/978-3-031-59468-7_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-59468-7_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-59467-0

  • Online ISBN: 978-3-031-59468-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation