Skip to main content
SpringerLink
Log in

Model-based testing leveraged for automated web tests

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

Agile methods and their practices have fostered the widespread presence of automated test cases. Such test cases have been successfully and extensively adopted to test different software levels, from unit tests (e.g., JUnit) to end-to-end Web Graphical User Interface (GUI) tests (e.g., Selenium Webdriver). While test execution is mostly automated by existing tools, test design remains a manual task. In the pursuit of a more automated test case generation approach, the use of models has been advocated by the model-based testing (MBT) technique. Current MBT approaches assume a top-down workflow in which testers design models, test cases are generated from models, and then scripts are written to automate test execution. Nevertheless, writing automated tests is nowadays a developer’s responsibility and testers face a scenario in which automated tests might be reused to produce new model-based tests. This paper aimed to improve the synergy between existing automated tests and MBT. To do so, we present an approach that infers a model from existing tests, a tester leverages this model to include new tests, and then test code is generated. We analyzed the proposed approach in the context of Web applications with system-level GUI tests that employ the Page Object pattern. For MBT, we adopted event-driven models augmented with parameters and test input data. We evaluated the approach’s applicability with a prototype tool called MoLeWe and an experimental study with nine open-source Web applications. The results provide some evidence that using MoLeWe may help to include new tests faster than manually coding them, while increasing the overall code coverage of the test suite.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. A partially concrete test invokes a mix of implemented/unimplemented methods, while a concrete test only calls implemented methods.

  2. https://jgrapht.org/

  3. https://w3techs.com/technologies/details/pl-php

  4. https://laravel.com/

  5. https://codeigniter.com/

  6. https://jquery.com/

  7. https://getbootstrap.com/

  8. https://plugins.jetbrains.com/plugin/93-metricsreloaded

  9. https://github.com/tarunlalwani/php-code-coverage-web

  10. In this study, this relates to the invocation of Page Objects’ methods.

  11. https://robotframework.org/

  12. A well-known tool to support BDD; please refer to https://cucumber.io.

  13. It is an XML-based language used to represent commands for Selenium.

  14. Gherkin is a textual language to specify scenarios in the Given-When-Then format.

References

  • Adamsen, C. Q., Mezzetti, G., & Møller, A. (2015). Systematic execution of Android test suites in adverse conditions. In The International Symposium on Software Testing and Analysis, ACM, pp. 83–93.

  • Anand, S., Burke, E. K., Chen, T. Y., Clark, J. A., Cohen, M. B., Grieskamp, W., et al. (2013). An orchestrated survey of methodologies for automated software test case generation. Journal of Systems and Software, 86(8), 1978–2001.

    Article  Google Scholar 

  • Angluin, D. (1982). Inference of reversible languages. Journal of ACM, 29(3), 741–765.

    Article  MathSciNet  Google Scholar 

  • Beck, K. L. (2003). Test-driven Development - by example. The Addison-Wesley signature series: Addison-Wesley.

    Google Scholar 

  • Beizer, B. (1990). Software Testing Techniques (2nd ed.). New York, NY, USA: Van Nostrand Reinhold Co.

    MATH  Google Scholar 

  • Belli, F., Beyazit, M., & Memon, A. M. (2012). Testing is an event-centric activity. In SERE (Companion), IEEE, pp. 198–206.

  • Belli, F., Budnik, C. J., & White, L. (2006). Event-based modelling, analysis and testing of user interactions: approach and case study. Software Testing, Verification and Reliability, 16(1), 3–32.

    Article  Google Scholar 

  • Bernardino, M., Rodrigues, E. M., Zorzo, A. F., & Marchezan, L. (2017). Systematic mapping study on MBT: tools and models. IET Software, 11(4), 141–155.

    Article  Google Scholar 

  • Chelimsky, D., Astels, D., Helmkamp, B., North, D., Dennis, Z., and Hellesoy, A. The RSpec Book: Behaviour Driven Development with RSpec, Cucumber, and Friends, 1st ed. Pragmatic Bookshelf, 2010.

  • Danglot, B., Vera-Perez, O., Yu, Z., Zaidman, A., Monperrus, M., & Baudry, B. (2019). A snowballing literature study on test amplification. Journal of Systems and Software, 157.

  • Danglot, B., Vera-Pérez, O. L., Baudry, B., & Monperrus, M. (2019). Automatic test improvement with DSpot: A study with ten mature open-source projects. Empirical Software Engineering, 1–33.

  • Dixit, R., Lutteroth, C., & Weber, G. (2015). FormTester: Effective integration of model-based and manually specified test cases. In 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering, pp. 745–748.

  • Entin, V., Winder, M., Zhang, B., & Christmann, S. (2011). Combining model-based and capture-replay testing techniques of graphical user interfaces: An industrial approach. In IEEE 4th International Conference on Software Testing, Verification and Validation Workshops, pp. 572–577.

  • Entin, V., Winder, M., Zhang, B., & Christmann, S. (2012). Introducing model-based testing in an industrial Scrum project. In 2012 7th International Workshop on Automation of Software Test (AST), pp. 43–49.

  • Entin, V., Winder, M., Zhang, B., & Claus, A. (2015). A process to increase the model quality in the context of model-based testing. In 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–7.

  • Fowler, M. (2013). PageObject. https://martinfowler.com/bliki/PageObject.html

  • Fraser, G., & Zeller, A. (2011). Generating parameterized unit tests. In Proceedings of the 2011 International Symposium on Software Testing and Analysis (ISSTA) (New York, NY, USA), pp. 364–374.

  • Grieskamp, W., Kicillof, N., Stobie, K., & Braberman, V. (2011). Model-based quality assurance of protocol documentation: Tools and methodology. Software Testing, Verification and Reliability, 21(1), 55–71.

    Article  Google Scholar 

  • Kazman, R., Woods, S. S., & Carrière, S. J. (1998). Requirements for integrating software architecture and reengineering models: CORUM II. In 5th Working Conference on Reverse Engineering, WCRE ’98, Honolulu, Hawai, USA, October 12-14, 1998, pp. 154–163.

  • Kramer, A., & Legeard, B. (2016). Model-Based Testing Essentials - Guide to the ISTQB Certified Model-Based Tester: Foundation Level, 1st ed. Wiley Publishing.

  • Kramer, A., Legeard, B., & Binder, R. V. (2017). 2016/2017 model-based testing user survey. http://www.cftl.fr/wp-content/uploads/2017/02/2016-MBT-User-Survey-Results.pdf

  • Leotta, M., Biagiola, M., Ricca, F., Ceccato, M., & Tonella, P. (2020). A family of experiments to assess the impact of page object pattern in web test suite development. In 13th IEEE International Conference on Software Testing, Validation and Verification, ICST 2020, Porto, Portugal, October 24-28, 2020, IEEE, pp. 263–273.

  • Leotta, M., Clerissi, D., Ricca, F., and Spadaro, C. (2013). Improving test suites maintainability with the page object pattern: An industrial case study. In Sixth IEEE International Conference on Software Testing, Verification and Validation, ICST 2013 Workshops Proceedings, Luxembourg, Luxembourg, March 18-22, 2013, IEEE Computer Society, pp. 108–113.

  • Li, N., Escalona, A., & Kamal, T. (2016). Skyfire: Model-based testing with Cucumber. In 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST), pp. 393–400.

  • Marchetto, A., Tonella, P., & Ricca, F. (2012). ReAjax: a reverse engineering tool for ajax web applications. IET Software, 6(1), 33–49.

    Article  Google Scholar 

  • Memon, A. M. (2007). An event-flow model of GUI-based applications for testing. Software Testing, Verification and Reliability, 17(3), 137–157.

    Article  Google Scholar 

  • Milani Fard, A., Mirzaaghaei, M., & Mesbah, A. (2014). Leveraging existing tests in automated test generation for web applications. In Proceedings of the 29th ACM/IEEE international conference on Automated software engineering, ACM, pp. 67–78.

  • Myers, G. J., Sandler, C., & Badgett, T. (2011). The Art of Software Testing, 3rd ed. Wiley Publishing.

  • Offutt, J., & Abdurazik, A. (1999). Generating tests from UML specifications. In: «UML»’99: The Unified Modeling Language - Beyond the Standard, Springer-Verlag, Berlin, Heidelberg, pp 416–429.

  • Orso, A., & Rothermel, G. (2014). Software testing: A research travelogue (2000–2014). In Proceedings of the on Future of Software Engineering (FOSE), ACM, pp. 117–132.

  • Pawlak, R., Monperrus, M., Petitprez, N., Noguera, C., & Seinturier, L. (2016). SPOON: A library for implementing analyses and transformations of java source code. Software: Practice and Experience, 46(9), 1155–1179.

    Google Scholar 

  • Peterson, K. (2013). The github open source development process. Mayo Clinic: Technical report.

    Google Scholar 

  • Ricca, F., Leotta, M., & Stocco, A. (2019). Chapter three - three open problems in the context of E2E web testing and a vision: NEONATE. Advances in Computers, 113, 89–133.

    Article  Google Scholar 

  • Schulze, C., Lindvall, M., Bjorgvinsson, S., & Wiegand, R. (2015). Model generation to support model-based testing applied on the NASA DAT web-application - An experience report. In 26th International Symposium on Software Reliability Engineering (ISSRE), pp. 77–87.

  • Spring.io. (2014). Spring PetClinic sample application. https://github.com/spring-projects/spring-petclinic

  • Stewart, S. (2015). Page objects. https://github.com/SeleniumHQ/selenium/wiki/PageObjects

  • Stocco, A., Leotta, M., Ricca, F., & Tonella, P. (2017). APOGEN: Automatic page object generator for web testing. Software Quality Journal, 25(3), 1007–1039.

    Article  Google Scholar 

  • Torens, C., Ebrecht, L., & Lemmer, K. (2011). Starting model-based testing based on existing test cases used for model creation. In 2011 IEEE 11th International Conference on Computer and Information Technology, pp. 320–327.

  • Utting, M., & Legeard, B. (2007). Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann Publishers Inc.

    Google Scholar 

  • Xie, T., & Notkin, D. (2006). Tool-assisted unit-test generation and selection based on operational abstractions. Automated Software Engineering, 13(3), 345–371.

    Article  Google Scholar 

  • Zander, J., Schieferdecker, I., & Mosterman, P. J. (2011). Model-Based Testing for Embedded Systems, 1st ed. CRC Press Inc.

  • Zhang, P., & Elbaum, S. (2014). Amplifying tests to validate exception handling code: An extended study in the mobile application domain. ACM Transactions on Software Engineering and Methodology, 23(4), 32:1–32:28.

Download references

Acknowledgements

Special thanks to the students and practitioners who took part in the empirical study. The authors are also grateful to the anonymous reviewers for their useful and comprehensive comments and suggestions that helped to strengthen the results. André T. Endo is partially financially supported by CNPq/Brazil (grant no. 420363/20181).

Author information

Authors and Affiliations

  1. Federal University of Technology - Parana, Cornelio Procopio, Brazil

    Guilherme Ricken Mattiello & André Takeshi Endo

Authors
  1. Guilherme Ricken Mattiello
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. André Takeshi Endo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to André Takeshi Endo.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mattiello, G.R., Endo, A.T. Model-based testing leveraged for automated web tests. Software Qual J 30, 621–649 (2022). https://doi.org/10.1007/s11219-021-09575-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-021-09575-w

Keywords

Search

Navigation