Abstract
Since only a limited time is available for performing software regression testing, a subset of crucial test cases from the test suites has to be selected for execution. In this paper, we introduce a method that uses the relation between types of code changes and regression tests to select test types that require execution. We work closely with a large power supply company to develop and evaluate the method and measure the total regression testing time taken by our method and its effectiveness in selecting the most relevant test types. The results show that the method reduces the total regression time by an average of 18,33% when compared with the approach used by our industrial partner. The results also show that using a medium window size in the method configuration results in an improved recall rate from 61,11% to 83,33%, but not in considerable time reduction of testing. We conclude that our method can potentially be used to steer the testing effort at software development companies by guiding testers into which regression test types are essential for execution.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
BoW is essentially a sequence of words/tokens, which are descendingly ordered according to frequency. This cut-off parameters controls how many of the most frequently used words are included as features – e.g. 10 means that the 10 most frequently used words become features and the rest are ignored.
- 2.
Due to non-disclosure agreements with our industrial partner, our data-set can not be made public for replication.
References
Iso/iec/ieee int. standard - software and systems engineering - software testing-part 1: Concepts and definitions. Technical report (2020)
Al-Sabbagh, K., Staron, M., Hebig, R., Gomes, F.: A classification of code changes and test types dependencies for improving machine learning based test selection. In: Proceedings of the 17th International Conference on Predictive Models and Data Analytics in Software Engineering, pp. 40–49 (2021)
Al-Sabbagh, K.W., Staron, M., Hebig, R., Meding, W.: Predicting test case verdicts using textual analysis of committed code churns. In: (IWSM Mensura 2019), vol. 2476, pp. 138–153 (2019)
Batista, G.E., Prati, R.C., Monard, M.C.: A study of the behavior of several methods for balancing machine learning training data. ACM SIGKDD Explorations Newsl 6(1), 20–29 (2004)
Bayerl, P.S., Paul, K.I.: What determines inter-coder agreement in manual annotations? a meta-analytic investigation. Comput. Linguist. 37(4), 699–725 (2011)
Bertolino, A., Guerriero, A., Miranda, B., Pietrantuono, R., Russo, S.: Learning-to-rank vs ranking-to-learn: strategies for regression testing in continuous integration. In: Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, pp. 1–12 (2020)
Chi, J., Qu, Y., Zheng, Q., Yang, Z., Jin, W., Cui, D., Liu, T.: Relation-based test case prioritization for regression testing. J. Syst. Softw. 163, 110539 (2020)
Dahiya, O., Solanki, K.: A systematic literature study of regression test case prioritization approaches. Int. J. Eng. Technol. 7(4), 2184–2191 (2018)
Durelli, V.H., et al.: Machine learning applied to software testing: a systematic mapping study. IEEE Trans. Reliab. 68(3), 1189–1212 (2019)
Esuli, A., Sebastiani, F.: Proceedings of the 5th conference on language resources and evaluation (2006)
Knauss, E., Staron, M., Meding, W., Söder, O., Nilsson, A., Castell, M.: Supporting continuous integration by code-churn based test selection. In: 2015 IEEE/ACM 2nd International Workshop on Rapid Continuous Software Engineering, pp. 19–25. IEEE (2015)
Ochodek, M., Staron, M., Bargowski, D., Meding, W., Hebig, R.: Using machine learning to design a flexible loc counter. In: 2017 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation, pp. 14–20. IEEE (2017)
de Oliveira Neto, F.G., Ahmad, A., Leifler, O., Sandahl, K., Enoiu, E.: Improving continuous integration with similarity-based test case selection. In: Proceedings of the 13th International Workshop on Automation of Software Test, pp. 39–45 (2018)
Orso, A., Shi, N., Harrold, M.J.: Scaling regression testing to large software systems. ACM SIGSOFT Soft. Eng. Notes 29(6), 241–251 (2004)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Wang, K., Zhu, C., Celik, A., Kim, J., Batory, D., Gligoric, M.: Towards refactoring-aware regression test selection. In: 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE), pp. 233–244. IEEE (2018)
Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in software engineering. Springer Science & Business Media (2012)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Al-Sabbagh, K.W., Staron, M., Hebig, R. (2022). Improving Software Regression Testing Using a Machine Learning-Based Method for Test Type Selection. In: Taibi, D., Kuhrmann, M., Mikkonen, T., Klünder, J., Abrahamsson, P. (eds) Product-Focused Software Process Improvement. PROFES 2022. Lecture Notes in Computer Science, vol 13709. Springer, Cham. https://doi.org/10.1007/978-3-031-21388-5_33
Download citation
DOI: https://doi.org/10.1007/978-3-031-21388-5_33
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21387-8
Online ISBN: 978-3-031-21388-5
eBook Packages: Computer ScienceComputer Science (R0)