Abstract
Mutation analysis is one of the most popular testing techniques. However, this technique has some drawbacks in terms of time and resource consumption. Higher order mutation has been recently an object of interest by research community for overcoming such issues by reducing the number of equivalent mutants, generated mutants and simulating better realistic faults. In this paper, we focus on two problems for higher-order mutation. The first one is to evaluate the quality of mutation operators as well as generated mutants. The second one is to prioritize test cases by means of capability of killing mutants. We propose the evaluation of mutation operators basing on first-order mutants, and the quality operators are selected to create higher-order mutants. We assess also the quality of each test case and rank the test cases for testing. The results may help developers optimize their resource during planning testing activities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lipton, R.: Fault diagnosis of computer programs. Student report, Carnegie Mellon University (1971)
DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Computer 11(4), 34–41 (1978)
Jia, Y., Harman, M.: Higher order mutation testing. Inf. Softw. Technol. 51(10), 1379–1393 (2009)
Harman, M., Jia, Y., Langdon, W.B.: A manifesto for higher order mutation testing. In: Third International Conference on Software Testing, Verification, and Validation Workshops (2010)
Polo, M., Piattini, M., Garcia-Rodriguez, I.: Decreasing the cost of mutation testing with second-order mutants. Softw. Test. Verification Reliab. 19(2), 111–131 (2008)
Papadakis, M., Malevris, N.: An empirical evaluation of the first and second order mutation testing strategies. In: Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation, Workshops ICSTW 2010, pp. 90–99. IEEE Computer Society (2010)
Kintis, M., Papadakis, M., Malevris, N.: Evaluating mutation testing alternatives: a collateral experiment. In: 17th Asia Pacific Software Engineering Conference, APSEC (2010)
Madeyski, L., Orzeszyna, W., Torkar, R., Józala, M.: Overcoming the equivalent mutant problem: a systematic literature review and a comparative experiment of second order mutation. IEEE Trans. Softw. Eng. 40(1), 23–44 (2014)
Langdon, W.B., Harman, M., Jia, Y.: Multi-objective higher order mutation testing with genetic programming. In: Proceedings Fourth Testing: Academic and Industrial Conference Practice and Research (2009)
Langdon, W.B., Harman, M., Jia, Y.: Efficient multi-objective higher order mutation testing with genetic programming. J. Syst. Softw. 83, 2416–2430 (2010)
Belli, F., Güler, N., Hollmann, A., Suna, G., Yıldız, E.: Model-based higher-order mutation analysis. In: Kim, T.-h., Kim, H.-K., Khan, M.K., Kiumi, A., Fang, W.-c., Ślęzak, D. (eds.) ASEA 2010. CCIS, vol. 117, pp. 164–173. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17578-7_17
Akinde, A.O.: Using higher order mutation for reducing equivalent mutants in mutation testing. Asian J. Comput. Sci. Inf. Technol. 2(3), 13–18 (2012)
Omar, E., Ghosh, S.: An exploratory study of higher order mutation testing in aspect-oriented programming. In: IEEE 23rd International Symposium on Software Reliability Engineering (2012)
Ghiduk, A.S.: Using evolutionary algorithms for higher-order mutation testing. IJCSI Int. J. Comput. Sci. 11(2), 93–104 (2014)
Nguyen, Q., Madeyski, L.: Searching for strongly subsuming higher order mutants by applying multi-objective optimization algorithm. In: Le Thi, H.A., Nguyen, N.T., Do, T.V. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 358, pp. 391–402. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-17996-4_35
Nguyen, Q.V., Madeyski, L.: Higher order mutation testing to drive development of new test cases: an empirical comparison of three strategies. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, T.-P. (eds.) ACIIDS 2016. LNCS (LNAI), vol. 9621, pp. 235–244. Springer, Heidelberg (2016). https://doi.org/10.1007/978-3-662-49381-6_23
Nguyen, Q.V., Madeyski, L.: Addressing mutation testing problems by applying multi-objective optimization algorithms and higher order mutation. J. Intell. Fuzzy Syst. 32(2), 1173–1182 (2017)
Van Nho, D., Vu, N.Q., Binh, N.T.: A solution for improving the effectiveness of higher order mutation testing. In: 2019 IEEE-RIVF International Conference on Computing and Communication Technologies (RIVF), Danang, Vietnam (2019)
Madeyski, L., Radyk, N.: Judy – a mutation testing tool for Java. IET Softw. 4(1), 32–42 (2010)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Do, VN., Nguyen, QV., Nguyen, TB. (2021). Evaluating Mutation Operator and Test Case Effectiveness by Means of Mutation Testing. In: Nguyen, N.T., Chittayasothorn, S., Niyato, D., Trawiński, B. (eds) Intelligent Information and Database Systems. ACIIDS 2021. Lecture Notes in Computer Science(), vol 12672. Springer, Cham. https://doi.org/10.1007/978-3-030-73280-6_66
Download citation
DOI: https://doi.org/10.1007/978-3-030-73280-6_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73279-0
Online ISBN: 978-3-030-73280-6
eBook Packages: Computer ScienceComputer Science (R0)