Abstract
Testing web applications through the GUI can be complex and time-consuming, as it involves checking the functionality of the system under test (SUT) from the user’s perspective. Random testing can improve test efficiency by automating the process, but achieving good exploration is difficult because it requires uniform distribution over a large search space while also taking into account the dynamic content commonly found in web applications. Reinforcement learning can improve the efficiency of random testing by guiding the generation of test sequences. This is achieved by assigning rewards to specific actions and using them to determine which actions are most likely to lead to a desired outcome. While rewards based on the difference between consecutive states are commonly used in modern tools, they can lead to the Jumping Between States (JBS) problem, where large rewards are generated without significantly increasing exploration. We propose a solution to the JBS problem by combining rewards based on the change of state and a metric to estimate the level of exploration reached in the next state based on the frequency of actions executed. Our results show that this multi-faceted approach increases the exploration efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
References
The (google) monkey (2010). https://developer.android.com/studio/test/monkey.html. Accessed 27 Jan 2023
Adamo, D., Khan, M.K., Koppula, S., Bryce, R.: Reinforcement learning for android GUI testing. In: 9th ACM A-TEST Workshop (2018)
Alégroth, E., Feldt, R., Kolström, P.: Maintenance of automated test suites in industry: an empirical study on visual GUI testing. IST 73, 66–80 (2016)
Arcuri, A., Briand, L.: A practical guide for using statistical tests to assess randomized algorithms in software engineering. In: 33rd ICSE. ACM (2011)
Borges, N., Hotzkow, J., Zeller, A.: Droidmate-2: a platform for android test generation. In: 33rd ACM/IEEE ASE, pp. 916–919 (2018)
Collins, E., Neto, A., Vincenzi, A., Maldonado, J.: Deep reinforcement learning based android application GUI testing. In: SBES, SBES 2021. ACM (2021)
Coppola, R., Ardito, L., Torchiano, M., Morisio, M.: Mobile testing: new challenges and perceived difficulties from developers of the italian industry. IT PROFESSIONAL (2019)
Degott, C., Borges, Jr., N.P., Zeller, A.: Learning user interface element interactions. In: 28th ACM SIGSOFT, ISSTA 2019, pp. 296–306. ACM (2019)
Grechanik, M., Xie, Q., Fu, C.: Creating GUI testing tools using accessibility technologies. In: ICST Workshops, pp. 243–250. IEEE (2009)
Gu, T., et al.: Aimdroid: activity-insulated multi-level automated testing for android applications. In: ICSME (2017)
Jabbarvand, R., Lin, J.-W., Malek, S.: Search-based energy testing of android. In: IEEE/ACM 41st International Conference on Software Engineering (ICSE)
Koroglu, Y., et al.: QBE: qlearning-based exploration of android applications. In: ICST (2018)
Mariani, L., Pezze, M., Riganelli, O., Santoro, M.: Autoblacktest: automatic black-box testing of interactive applications. In: 5th ICST (2012)
Nyman, N.: Using monkey test tools - how to find bugs cost-effectively through random testing. Software Testing & Quality Engineering, Jan/Feb:18–21 (2000)
Pan, M., Huang, A., Wang, G., Zhang, T., Li, X.: Reinforcement learning based curiosity-driven testing of android applications. In: 29th SIGSOFT. ASM (2020)
Patel, P., Srinivasan, G., Rahaman, S., Neamtiu, I.: On the effectiveness of random testing for android. In A-TEST (2018)
Robbins, H.: Some aspects of the sequential design of experiments. Bull. Am. Math. Soc. 58(5), 527–535 (1952)
Rodríguez-Valdés, O., Vos, T.E.J., Aho, P., Marín, B.: 30 years of automated GUI testing: a bibliometric analysis. In: Paiva, A.C.R., Cavalli, A.R., Ventura Martins, P., Pérez-Castillo, R. (eds.) QUATIC 2021. CCIS, vol. 1439, pp. 473–488. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-85347-1_34
Romdhana, A., Merlo, A., Ceccato, M., Tonella, P.: Deep reinforcement learning for black-box testing of android apps (2021). arXiv preprint arXiv:2101.02636
Sutton, R.S., Barto, A.G.: Introduction to Reinforcement Learning, 1st edn. MIT Press, Cambridge (1998)
Vos, T.E., Aho, P., Pastor Ricos, F., Rodriguez-Valdes, O., Mulders, A.: TESTAR - scriptless testing through graphical user interface. STVR 31(3), e1771 (2021)
Vuong, T., Takada, S.: A reinforcement learning based approach to automated testing of android applications. In: 9th ACM A-TEST Workshop (2018)
Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3), 279–292 (1992)
Wiering, M.A., Van Otterlo, M.: Reinforcement learning. Adapt. Learn. Optim. 12(3), 729 (2012)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer Science & Business Media, New York (2012). https://doi.org/10.1007/978-1-4615-4625-2
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rodríguez-Valdés, O., Vos, T.E.J., Marín, B., Aho, P. (2023). Reinforcement Learning for Scriptless Testing: An Empirical Investigation of Reward Functions. In: Nurcan, S., Opdahl, A.L., Mouratidis, H., Tsohou, A. (eds) Research Challenges in Information Science: Information Science and the Connected World. RCIS 2023. Lecture Notes in Business Information Processing, vol 476. Springer, Cham. https://doi.org/10.1007/978-3-031-33080-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-33080-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-33079-7
Online ISBN: 978-3-031-33080-3
eBook Packages: Computer ScienceComputer Science (R0)