Abstract
The IPO (In-Parameter-Order) family of algorithms is a popular set of greedy methods for the construction of covering arrays. Aspects such as tie-breaking behavior or parameter ordering can have major impact on the quality of the resulting arrays but have so far not been studied in a systematic manner. In this paper, we survey and present a general framework for the IPO family of algorithms (i.e. IPOG, IPOG-F and IPOG-F2) and present ways to instantiate these abstract components. Then, we evaluate the performance of these variations on a large set of instances, in an extensive experimental setting in terms of covering array sizes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bryce, R.C., Colbourn, C.J., Cohen, M.B.: A framework of greedy methods for constructing interaction test suites. In: Proceedings of the 27th International Conference on Software Engineering, ICSE 2005, pp. 146–155. ACM (2005)
Cheng, C.T.: The test suite generation problem: optimal instances and their implications. Discrete Appl. Math. 155(15), 1943–1957 (2007)
Cohen, M.B., Gibbons, P.B., Mugridge, W.B., Colbourn, C.J.: Constructing test suites for interaction testing. In: Proceedings of the 25th International Conference on Software Engineering, ICSE 2003, pp. 38–48. IEEE Computer Society (2003)
Duan, F., Lei, Y., Yu, L., Kacker, R.N., Kuhn, D.R.: Improving IPOG’s vertical growth based on a graph coloring scheme. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–8 (2015)
Forbes, M., Lawrence, J., Lei, Y., Kacker, R.N., Kuhn, D.R.: Refining the in-parameter-order strategy for constructing covering arrays. J. Res. Nat. Inst. Stan. Technol. 113(5), 287 (2008)
Gao, S.W., Lv, J.H., Du, B.L., Colbourn, C.J., Ma, S.L.: Balancing frequencies and fault detection in the in-parameter-order algorithm. J. Comput. Sci. Technol. 30(5), 957–968 (2015)
Gao, S., Lv, J., Du, B., Jiang, Y., Ma, S.: General optimization strategies for refining the in-parameter-order algorithm. In: 2014 14th International Conference on Quality Software (QSIC), pp. 21–26. IEEE (2014)
Hartman, A.: Software and hardware testing using combinatorial covering suites. In: Golumbic, M., Hartman, I.A. (eds.) Graph Theory, Combinatorics and Algorithms, Operations Research/Computer Science Interfaces Series, vol. 34, pp. 237–266. Springer, Heidelberg (2005)
Hartman, A., Raskin, L.: Problems and algorithms for covering arrays. Discrete Math. 284(1–3), 149–156 (2004)
Kampel, L., Simos, D.E.: Set-based algorithms for combinatorial test set generation. In: Wotawa, F., Nica, M., Kushik, N. (eds.) ICTSS 2016. LNCS, vol. 9976, pp. 231–240. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47443-4_16
Kleine, K., Simos, D.E.: An efficient design and implementation of the in-parameter-order algorithm. Math. Comput. Sci. 12(1), 51–67 (2018)
Kuhn, D., Kacker, R., Lei, Y.: Introduction to Combinatorial Testing. Chapman & Hall/CRC Innovations in Software Engineering and Software Development Series. Taylor & Francis, London (2013)
Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG: a general strategy for T-way software testing. In: 14th Annual IEEE International Conference and Workshops on the Engineering of Computer-Based Systems, ECBS 2007, pp. 549–556. IEEE (2007)
Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing. Softw. Test. Verification Reliab. 18(3), 125–148 (2008)
Lei, Y., Tai, K.C.: In-parameter-order: a test generation strategy for pairwise testing. In: Proceedings of Third IEEE International High-Assurance Systems Engineering Symposium, pp. 254–261. IEEE (1998)
Younis, M.I., Zamli, K.Z.: MIPOG-an efficient t-way minimization strategy for combinatorial testing. Int. J. Comput. Theory Eng. 3(3), 388 (2011)
Yu, L., Duan, F., Lei, Y., Kacker, R.N., Kuhn, D.R.: Constraint handling in combinatorial test generation using forbidden tuples. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–9 (2015)
Yu, L., Lei, Y., Nourozborazjany, M., Kacker, R.N., Kuhn, D.R.: An efficient algorithm for constraint handling in combinatorial test generation. In: 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation, pp. 242–251 (2013)
Acknowledgments
The research presented in this paper has been funded in part by the Austrian Research Promotion Agency (FFG) under grants 851205 (Security ProtocoL Interaction Testing in Practice - SPLIT) and 865248 (SECuring Web technologies with combinatorial Interaction Testing - SecWIT).
Part of this research has also been carried out in the context of the Austrian COMET K1 program which is publicly funded by the Austrian Research Promotion Agency (FFG) and the Vienna Business Agency (WAW).
This work was made possible by the facilities of the Shared Hierarchical Academic Research Computing Network (SHARCNET) and Compute/Calcul Canada.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Kleine, K., Kotsireas, I., Simos, D.E. (2018). Evaluation of Tie-Breaking and Parameter Ordering for the IPO Family of Algorithms Used in Covering Array Generation. In: Iliopoulos, C., Leong, H., Sung, WK. (eds) Combinatorial Algorithms. IWOCA 2018. Lecture Notes in Computer Science(), vol 10979. Springer, Cham. https://doi.org/10.1007/978-3-319-94667-2_16
Download citation
DOI: https://doi.org/10.1007/978-3-319-94667-2_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-94666-5
Online ISBN: 978-3-319-94667-2
eBook Packages: Computer ScienceComputer Science (R0)