Abstract
Test automation brings several benefits but also presents significant problems that often force developers/testers to carry out tiring and costly manual tasks. Among these tiring tasks while using Selenium WebDriver for testing Web applications, driver management (i.e., version discovery, download, setup and maintenance) ranks in a top position.
Recently, an open source Java library, named WebDriverManager, that carries out automatically the driver management process for Selenium WebDriver has been proposed to alleviate the burden of the developers. This library appears to be very promising but until now no one has experimentally evaluated its effectiveness.
In this paper, we present an empirical study aimed at understanding whether the use of WebDriverManager allows to reduce both initial and long-term setup efforts of a multibrowser test suite.
The results are in favor of WebDriverManager as it allows an average saving of more than 33% of the time for an initial setup which translated annually corresponds to many hours of bare manual driver management to which must be added the hours for understanding that the outdated driver is the cause of the problems reported by the test suite. Having almost weekly, as we estimated, outdated drivers brings also to a reduction in the perceived reliability of E2E test suites. The adoption of WebDriverManager can help to drastically reduce all these problems.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
- 2.
- 3.
- 4.
- 5.
- 6.
- 7.
- 8.
Note that the setup of different drivers has approximately the same effort, so adopting different browsers/Drivers do not impact the results of our study.
- 9.
- 10.
- 11.
- 12.
References
Ardito, L., Coppola, R., Morisio, M., Torchiano, M.: Espresso vs. eyeautomate: an experiment for the comparison of two generations of android gui testing. In: Proceedings of the Evaluation and Assessment on Software Engineering, EASE 2019, pp. 13–22. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3319008.3319022
Basili, V.R., Caldiera, G., Rombach, H.D.: The goal question metric approach. In: Encyclopedia of Software Engineering, Wiley, Hoboken (1994)
Cerioli, M., Lagorio, G., Leotta, M., Ricca, F.: Fight silent horror unit test methods by consulting a TestWizard. J. Softw. Evol. Proc. (JSEP) e2396 (2022). https://doi.org/10.1002/smr.2396
Cerioli, M., Leotta, M., Ricca, F.: What 5 million job advertisements tell us about testing: a preliminary empirical investigation. In: Proceedings of the 35th Annual ACM Symposium on Applied Computing, pp. 1586–1594 (2020)
Clarke, S.: Measuring API usability. Dr. Dobb’s J. Windows/.NET Suppl. 9(5), S6–S9 (2004)
Garcia, B.: Mastering Software Testing with JUnit 5: Comprehensive Guide to Develop high Quality Java Applications. Packt Publishing Ltd, Birmingham (2017)
García, B.: Hands-On Selenium WebDriver with Java. O’Reilly Media, Sebastopol (2022)
García, B., Gallego, M., Gortázar, F., Munoz-Organero, M.: A survey of the selenium ecosystem. Electronics 9(7), 1067 (2020)
Garcia, B., Gortazar, F., Lopez-Fernandez, L., Gallego, M., Paris, M.: WebRTC testing: challenges and practical solutions. IEEE Commun. Stand. Mag. 1(2), 36–42 (2017)
García, B., Kloos, C.D., Alario-Hoyos, C., Munoz-Organero, M.: Selenium-jupiter: a JUnit 5 extension for selenium WebDriver. J. Syst. Softw. 189, 111298 (2022)
García, B., Munoz-Organero, M., Alario-Hoyos, C., Kloos, C.D.: Automated driver management for selenium WebDriver. Empir. Softw. Eng. 26(5), 1–51 (2021). https://doi.org/10.1007/s10664-021-09975-3
Grissom, R.J., Kim, J.J.: Effect sizes for Research: A Broad Practical Approach. 2nd edn. Lawrence Earlbaum Associates, Mahwah (2005)
Kirinuki, H., Matsumoto, S., Higo, Y., Kusumoto, S.: NLP-assisted web element identification toward script-free testing. In: 2021 IEEE International Conference on Software Maintenance and Evolution (ICSME), pp. 639–643 (2021). https://doi.org/10.1109/ICSME52107.2021.00072
Leotta, M., Biagiola, M., Ricca, F., Ceccato, M., Tonella, P.: A family of experiments to assess the impact of page object pattern in web test suite development. In: Proceedings of 13th IEEE International Conference on Software Testing, Verification and Validation (ICST 2020), pp. 263–273. IEEE (2020). https://doi.org/10.1109/ICST46399.2020.00035
Leotta, M., Cerioli, M., Olianas, D., Ricca, F.: Two experiments for evaluating the impact of Hamcrest and AssertJ on assertion development. Software Qual. J. 28(3), 1113–1145 (2020). https://doi.org/10.1007/s11219-020-09507-0
Motulsky, H.: Intuitive Biostatistics: a Non-Mathematical Guide to Statistical Thinking. Oxford University Press, Oxford (2010)
Naidu, T.J., Basri, N.A., Nagenthram, S.: SAHI vs. selenium: a comparative analysis. In: 2014 International Conference on Contemporary Computing and Informatics (IC3I), pp. 967–970 (2014). https://doi.org/10.1109/IC3I.2014.7019594
Olianas, D., Leotta, M., Ricca, F., Villa, L.: Reducing flakiness in end-to-end test suites: an experience report. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds.) QUATIC 2021. CCIS, vol. 1439, pp. 3–17. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85347-1_1
Phuc Nguyen, D., Maag, S.: Codeless web testing using selenium and machine learning. In: ICSOFT 2020: 15th International Conference on Software Technologies, pp. 51–60. 15th International Conference on Software and Data Technologies (ICSOFT), ScitePress, France, July 2020. https://doi.org/10.5220/0009885400510060
Ricca, F., Torchiano, M., Leotta, M., Tiso, A., Guerrini, G., Reggio, G.: On the impact of state-based model-driven development on maintainability: a family of experiments using UniMod. Empir. Softw. Eng. 23(3), 1743–1790 (2017). https://doi.org/10.1007/s10664-017-9563-8
Sazoglu, F.B., Cambazoglu, B.B., Ozcan, R., Altingovde, I.S., Ulusoy, Ö.: Strategies for setting time-to-live values in result caches. In: Proceedings of the 22nd ACM International Conference on Information & Knowledge Management, pp. 1881–1884 (2013)
Stewart, S., Burns, D.: WebDriver, W3C Working Draft. https://www.w3.org/TR/webdriver/ (2022). Accessed 1 Apr 2022
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M., Regnell, B., Wesslén, A.: Experimentation in Software Engineering - An Introduction. Kluwer Academic Publishers, Dordrecht (2000)
Acknowledgement
This work was partially supported in part by the Ministerio de Ciencia e Innovación-Agencia Estatal de Investigación (10.13039/501100011033) through the H2O Learn project under Grant PID2020-112584RB-C31, and in part by the Madrid Regional Government through the e-Madrid-CM Project under Grant S2018/TCS-4307.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Leotta, M., García, B., Ricca, F. (2022). An Empirical Study to Quantify the SetUp and Maintenance Benefits of Adopting WebDriverManager. In: Vallecillo, A., Visser, J., Pérez-Castillo, R. (eds) Quality of Information and Communications Technology. QUATIC 2022. Communications in Computer and Information Science, vol 1621. Springer, Cham. https://doi.org/10.1007/978-3-031-14179-9_3
Download citation
DOI: https://doi.org/10.1007/978-3-031-14179-9_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-14178-2
Online ISBN: 978-3-031-14179-9
eBook Packages: Computer ScienceComputer Science (R0)