Abstract
The development of a new software system involves extensive tests on the software functionality in order to identify possible failures. It will be ideal to test all possible input cases (configurations), but the exhaustive approach usually demands too large cost and time. The test suite reduction problem can be defined as the task of generating small set of test cases under certain requirements. A way to design test suites is through interaction testing using a matrix called Covering Array, CA(N;t,k,v), which guarantees that all configurations among every t parameters are covered. This paper presents a simple strategy that reduces the number of rows of a CA. The algorithms represent a post-optimization process which detects wild cards (values that can be changed arbitrarily without the CA losses its degree of coverage) and uses them to merge rows. In the experiment, 667 CAs, created by a state-of-the-art algorithm, were subject to the reduction process. The results report a reduction in the size of 347 CAs (52% of the cases). As part of these results, we report the matrix for CA(42;2,8,6) constructed from CA(57;2,8,6) with an impressive reduction of 15 rows, which is the best upper bound so far.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bracho-Rios, J., Torres-Jimenez, J., Rodriguez-Tello, E.: A New Backtracking Algorithm for Constructing Binary Covering Arrays of Variable Strength. In: Aguirre, A.H., Borja, R.M., Garciá, C.A.R. (eds.) MICAI 2009. LNCS, vol. 5845, pp. 397–407. Springer, Heidelberg (2009)
Bryce, R.C., Colbourn, C.J.: Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information and Software Technology 48(10), 960–970 (2006)
Bryce, R.C., Colbourn, C.J.: The density algorithm for pairwise interaction testing: Research articles. Software Testing, Verification and Reliability 17, 159–182 (2007)
Burnstein, I.: Practical software testing: a process-oriented approach. Springer Professional Computing (2003) ISBN: 0-387-95131-8
Changhai, N., Hareton, L.: A survey of combinatorial testing. ACM Computing Surveys (CSUR) 43, 11:1–11:29 (2011)
Colbourn, C.J.: Combinatorial aspects of covering arrays. Le Matematiche (Catania) 58, 121–167 (2004)
Colbourn, C.J.: Covering arrays from cyclotomy. Designs, Codes and Cryptography 55, 201–219 (2010)
Colbourn, C.J., Martirosyan, S., Trung, T., Walker II., R.A.: Roux-type constructions for covering arrays of strengths three and four. Designs, Codes and Cryptography 41, 33–57 (2006), doi:10.1007/s10623-006-0020-8
Colbourn, C.J., Martirosyan, S.S., Mullen, G.L., Shasha, D., Sherwood, G.B., Yucas, J.L.: Products of mixed covering arrays of strength two. Journal of Combinatorial Designs 14(2), 124–138 (2006)
Forbes, M., Lawrence, J., Lei, Y., Kacker, R.N., Kuhn, D.R.: Refining the in-parameter-order strategy for constructing covering arrays. Journal of Research of the National Institute of Standards and Technology 113(5), 287–297 (2008)
Gonzalez-Hernandez, L., Rangel-Valdez, N., Torres-Jimenez, J.: Construction of mixed covering arrays of strengths 2 through 6 using a tabu search approach. In: Discrete Mathematics, Algorithms and Applications, pp. 1–20 (to appear, 2012)
Hartman, A., Ben, I.: Graph Theory, combinatorics and algorithms interdisciplinary applications, 4th edn. Springer, New York (2005) ISBN 13:9780387243474
Ji, L., Yin, J.: Constructions of new orthogonal arrays and covering arrays of strength three. Journal of Combinatorial Theory Series A 117, 236–247 (2010)
Kuhn, D.R., Reilly, M.J.: An investigation of the applicability of design of experiments to software testing. In: Proceedings of the 27th Annual NASA Goddard Software Engineering Workshop (SEW-27’02), SEW 2002, pp. 91–95. IEEE Computer Society, Washington, DC (2002)
Kuhn, D.R., Wallance, D.R., Gallo Jr., A.M.: Software fault interactions and implications for software testing. IEEE Transactions on Software Engineering 30, 418–421 (2004)
Kuliamin, V., Petukhov, A.: A survey of methods for constructing covering arrays. Programming and Computer Software 37, 121–146 (2011)
Lawrence, J., Kacker, R.N., Lei, Y., Kuhn, D.R., Forbes, M.: A survey of binary covering arrays. Electronic Journals of Combinatorics 18, 84 (2011)
Levenson, N.G., Turner, C.S.: An investigation of the therac-25 accidents. IEEE Computer 26, 18–41 (1993)
Lions, J.L.: Ariane 5, flight 501, report of the inquiry board. European Space Agency (July 1996)
Nayeri, P., Colbourn, C.J., Konjevod, G.: Randomized Postoptimization of Covering Arrays. In: Fiala, J., Kratochvíl, J., Miller, M. (eds.) IWOCA 2009. LNCS, vol. 5874, pp. 408–419. Springer, Heidelberg (2009)
Sampath, S., Bryce, R.C.: Improving the effectiveness of test suite reduction for user-session-based testing of web applications. Information and Software Technology 54(7), 724–738 (2012)
Tassey, G.: The economic impacts of inadequate infrastructure for software testing. Technical report, National Institute of Standards and Technology (May 2002)
Torres-Jimenez, J., Rangel-Valdez, N., Gonzalez-Hernandez, L.: An exact approach to maximize the number of wild cards in a covering array. In: Submitted in Mexican International Conference on Artificial Intelligence, Puebla, México, November 26-December 4 (2011)
Torres-Jimenez, J., Rodriguez-Tello, E.: Simulated annealing for constructing binary covering arrays of variable strength. In: IEEE Congress on Evolutionary Computation CEC 2010, July 18-23, pp. 1–8 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Gonzalez-Hernandez, L., Torres-Jimenez, J., Rangel-Valdez, N., Bracho-Rios, J. (2013). A Post-optimization Strategy for Combinatorial Testing: Test Suite Reduction through the Identification of Wild Cards and Merge of Rows. In: Batyrshin, I., Mendoza, M.G. (eds) Advances in Computational Intelligence. MICAI 2012. Lecture Notes in Computer Science(), vol 7630. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-37798-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-37798-3_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-37797-6
Online ISBN: 978-3-642-37798-3
eBook Packages: Computer ScienceComputer Science (R0)