CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays
Authors: 
Qiyuan Zhao, Chuan Luo, Shaowei Cai, Wei Wu, Jinkun Lin, Hongyu Zhang, Chunming HuAuthors Info & Claims
Qiyuan Zhao, Chuan Luo, Shaowei Cai, Wei Wu, Jinkun Lin, Hongyu Zhang, Chunming HuAuthors Info & ClaimsPages 81 - 93
Abstract
The increasing demand for software customization has led to the development of highly configurable systems. Combinatorial interaction testing (CIT) is an effective method for testing these types of systems. The ultimate goal of CIT is to generate a test suite of acceptable size, called a t-wise covering array (CA), where t is the testing strength. Pairwise testing (i.e., CIT with t=2) is recognized to be the most widely-used CIT technique and has strong fault detection capability. In pairwise testing, the most important problem is pairwise CA generation (PCAG), which is to generate a pairwise CA (PCA) of minimum size. However, existing state-of-the-art PCAG algorithms suffer from the severe scalability challenge; that is, they cannot tackle large-scale PCAG instances effectively, resulting in PCAs of large sizes. To alleviate this challenge, in this paper we propose CAmpactor, a novel and effective local search algorithm for compacting given PCAs into smaller sizes. Extensive experiments on a large number of real-world, public PCAG instances show that the sizes of CAmpactor's generated PCAs are around 45% smaller than the sizes of PCAs constructed by existing state-of-the-art PCAG algorithms, indicating its superiority. Also, our evaluation confirms the generality of CAmpactor, since CAmpactor can reduce the sizes of PCAs generated by a variety of PCAG algorithms.
Supplementary Material
"The increasing demand for software customization has led to the development of highly configurable systems. Combinatorial interaction testing (CIT) is an effective method for testing these types of systems. The ultimate goal of CIT is to generate a test suite of acceptable size, called a t-wise covering array (CA), where t is the testing strength. Pairwise testing (i.e., CIT with t=2) is recognized to be the most widely-used CIT technique and has strong fault detection capability. In pairwise testing, the most important problem is pairwise CA generation (PCAG), which is to generate a pairwise CA (PCA) of minimum size. However, existing state-of-the-art PCAG algorithms suffer from the severe scalability challenge; that is, they cannot tackle large-scale PCAG instances effectively, resulting in PCAs of large sizes. To alleviate this challenge, in this paper we propose CAmpactor, a novel and effective local search algorithm for compacting given PCAs into smaller sizes. Extensive experiments on a large number of real-world, public PCAG instances show that the sizes of CAmpactor’s generated PCAs are around 45% smaller than the sizes of PCAs constructed by existing state-of-the-art PCAG algorithms, indicating its superiority. Also, our evaluation confirms the generality of CAmpactor, since CAmpactor can reduce the sizes of PCAs generated by a variety of PCAG algorithms."
- Download
- 67.53 MB
References
[1]
Carlos Ansótegui, Felip Manyà, Jesus Ojeda, Josep M. Salvia, and Eduard Torres. 2022. Incomplete MaxSAT approaches for combinatorial testing. Journal of Heuristics, 28, 4 (2022), 377–431.
[2]
Sven Apel, Don S. Batory, Christian Kästner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines - Concepts and Implementation. Springer.
[3]
Adrian Balint and Andreas Fröhlich. 2010. Improving Stochastic Local Search for SAT with a New Probability Distribution. In Proceedings of SAT 2010. Springer, 10–15.
[4]
Mutsunori Banbara, Haruki Matsunaka, Naoyuki Tamura, and Katsumi Inoue. 2010. Generating Combinatorial Test Cases by Efficient SAT Encodings Suitable for CDCL SAT Solvers. In Proceedings of LPAR 2010. 112–126.
[5]
Eduard Baranov, Axel Legay, and Kuldeep S. Meel. 2020. Baital: An Adaptive Weighted Sampling Approach for Improved t-wise Coverage. In Proceedings of ESEC/FSE 2020. 1114–1126.
[6]
Don S. Batory. 2005. Feature Models, Grammars, and Propositional Formulas. In Proceedings of SPLC 2005. 7–20.
[7]
2009. Handbook of Satisfiability, Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh (Eds.) (Frontiers in Artificial Intelligence and Applications, Vol. 185). IOS Press. isbn:978-1-58603-929-5
[8]
Renée C. Bryce and Charles J. Colbourn. 2007. The Density Algorithm for Pairwise Interaction Testing. Software Testing, Verification and Reliability, 17, 3 (2007), 159–182.
[9]
Renée C. Bryce and Charles J. Colbourn. 2009. A Density-based Greedy Algorithm for Higher Strength Covering Arrays. Software Testing, Verification and Reliability, 19, 1 (2009), 37–53.
[10]
Renée C. Bryce, Charles J. Colbourn, and Myra B. Cohen. 2005. A Framework of Greedy Methods for Constructing Interaction Test Suites. In Proceedings of ICSE 2005. 146–155.
[11]
Shaowei Cai, Jinkun Lin, and Chuan Luo. 2017. Finding A Small Vertex Cover in Massive Sparse Graphs: Construct, Local Search, and Preprocess. Journal of Artificial Intelligence Research, 59 (2017), 463–494.
[12]
Shaowei Cai and Kaile Su. 2013. Local search for Boolean Satisfiability with configuration checking and subscore. Artificial Intelligence, 204 (2013), 75–98.
[13]
Xinye Cai, Haoran Sun, Qingfu Zhang, and Yuhua Huang. 2019. A Grid Weighted Sum Pareto Local Search for Combinatorial Multi and Many-Objective Optimization. IEEE Transactions on Cybernetics, 49, 9 (2019), 3586–3598.
[14]
Yanshan Chen, Ziyuan Wang, Dong Wang, Chunrong Fang, and Zhenyu Chen. 2019. Variable Strength Combinatorial Testing for Deep Neural Networks. In Proceedings of ICST Workshops 2019. 281–284.
[15]
Yi Chu, Shaowei Cai, and Chuan Luo. 2023. NuWLS: Improving Local Search for (Weighted) Partial MaxSAT by New Weighting Techniques. In Proceedings of AAAI 2023. 3915–3923.
[16]
David M. Cohen, Siddhartha R. Dalal, Michael L. Fredman, and Gardner C. Patton. 1997. The AETG System: An Approach to Testing Based on Combinatorial Design. IEEE Transactions on Software Engineering, 23, 7 (1997), 437–444.
[17]
Myra B. Cohen, Charles J. Colbourn, and Alan C. H. Ling. 2003. Augmenting Simulated Annealing to Build Interaction Test Suites. In Proceedings of ISSRE 2003. 394–405.
[18]
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, and Charles J. Colbourn. 2003. Constructing Test Suites for Interaction Testing. In Proceedings ICSE 2003. 38–48.
[19]
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, Charles J. Colbourn, and James S. Collofello. 2003. A Variable Strength Interaction Testing of Components. In Proceedings of COMPAC 2003. 413–418.
[20]
W. J. Conover. 1999. Practical Nonparametric Statistics. Conover. isbn:Conover
[21]
Hang Dong, Boshi Wang, Bo Qiao, Wenqian Xing, Chuan Luo, Si Qin, Qingwei Lin, Dongmei Zhang, Gurpreet Virdi, and Thomas Moscibroda. 2021. Predictive Job Scheduling under Uncertain Constraints in Cloud Computing. In Proceedings of IJCAI 2021. 3627–3634.
[22]
Niklas Eén and Niklas Sörensson. 2003. An Extensible SAT-solver. In Proceedings of SAT 2003. 502–518.
[23]
Brady J. Garvin, Myra B. Cohen, and Matthew B. Dwyer. 2009. An Improved Meta-Heuristic Search for Constrained Interaction Testing. In Proceedings of International Symposium on Search Based Software Engineering 2009. 13–22.
[24]
Brady J. Garvin, Myra B. Cohen, and Matthew B. Dwyer. 2011. Evaluating Improvements to a Meta-heuristic Search for Constrained Interaction Testing. Empirical Software Engineering, 16, 1 (2011), 61–102.
[25]
Syed A. Ghazi and Moataz A. Ahmed. 2003. Pair-wise Test Coverage using Genetic Algorithms. In Proceedings of CEC 2003. 1420–1424.
[26]
Harrison Goldstein, John Hughes, Leonidas Lampropoulos, and Benjamin C. Pierce. 2021. Do Judge a Test by its Cover - Combining Combinatorial and Property-Based Testing. In Proceedings of ESOP 2021. 264–291.
[27]
Loreto Gonzalez-Hernandez, Nelson Rangel-Valdez, and Jose Torres-Jimenez. 2010. Construction of Mixed Covering Arrays of Variable Strength Using a Tabu Search Approach. In Proceedings of COCOA 2010. 51–64.
[28]
Jiazhen Gu, Chuan Luo, Si Qin, Bo Qiao, Qingwei Lin, Hongyu Zhang, Ze Li, Yingnong Dang, Shaowei Cai, Wei Wu, Yangfan Zhou, Murali Chintalapati, and Dongmei Zhang. 2020. Efficient Incident Identification from Multi-dimensional Issue Reports via Meta-heuristic Search. In Proceedings of ESEC/FSE 2020. 292–303.
[29]
Christopher Henard, Mike Papadakis, Gilles Perrouin, Jacques Klein, Patrick Heymans, and Yves Le Traon. 2014. Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines. IEEE Transactions on Software Engineering, 40, 7 (2014), 650–670.
[30]
Aymeric Hervieu, Dusica Marijan, Arnaud Gotlieb, and Benoit Baudry. 2016. Practical Minimization of Pairwise-covering Test Configurations using Constraint Programming. Information and Software Technology, 71 (2016), 129–146.
[31]
Brahim Hnich, Steven David Prestwich, Evgeny Selensky, and Barbara M. Smith. 2006. Constraint Models for the Covering Test Problem. Constraints, 11, 2-3 (2006), 199–219.
[32]
Holger H. Hoos. 2002. An Adaptive Noise Mechanism for WalkSAT. In Proceedings of AAAI 2002. 655–660.
[33]
Holger H. Hoos and Thomas Stützle. 2004. Stochastic Local Search: Foundations & Applications. Elsevier / Morgan Kaufmann. isbn:1-55860-872-9
[34]
Yue Jia, Myra B. Cohen, Mark Harman, and Justyna Petke. 2015. Learning Combinatorial Interaction Test Generation Strategies Using Hyperheuristic Search. In Proceedings of ICSE 2015. 540–550.
[35]
Christian Kaltenecker, Alexander Grebhahn, Norbert Siegmund, Jianmei Guo, and Sven Apel. 2019. Distance-based Sampling of Software Configuration Spaces. In Proceedings of ICSE 2019. 1084–1094.
[36]
Alexander Knüppel, Thomas Thüm, Stephan Mennicke, Jens Meinicke, and Ina Schaefer. 2017. Is There a Mismatch between Real-world Feature Models and Product-line Research? In Proceedings of ESEC/FSE 2017. 291–302.
[37]
Sebastian Krieter, Thomas Thüm, Sandro Schulze, Gunter Saake, and Thomas Leich. 2020. YASA: yet another sampling algorithm. In Proceedings of VaMoS 2020. 4:1–4:10.
[38]
D. Richard Kuhn, Raghu N. Kacker, and Yu Lei. 2013. Introduction to Combinatorial Testing. CRC press. isbn:978-1-466-55229-6
[39]
D. Richard Kuhn, Dolores R. Wallace, and Albert M. Gallo. 2004. Software Fault Interactions and Implications for Software Testing. IEEE Transactions on Software Engineering, 30, 6 (2004), 418–421.
[40]
Rick Kuhn, Raghu N. Kacker, Jeff Yu Lei, and Dimitris E. Simos. 2020. Input Space Coverage Matters. Computer, 53, 1 (2020), 37–44.
[41]
Yu Lei, Raghu Kacker, D. Richard Kuhn, Vadim Okun, and James Lawrence. 2007. IPOG: A General Strategy for T-Way Software Testing. In Proceedings of ECBS 2007. 549–556.
[42]
Yu Lei, Raghu Kacker, D. Richard Kuhn, Vadim Okun, and James Lawrence. 2008. IPOG/IPOG-D: Efficient Test Generation for Multi-way Combinatorial Testing. Software Testing, Verification and Reliability, 18, 3 (2008), 125–148.
[43]
Yu Lei and Kuo-Chung Tai. 1998. In-Parameter-Order: A Test Generation Strategy for Pairwise Testing. In Proceedings of HASE 1998. 254–261.
[44]
Chu Min Li and Wenqi Huang. 2005. Diversification and Determinism in Local Search for Satisfiability. In Proceedings of SAT 2005. 158–172.
[45]
Jia Hui Liang, Vijay Ganesh, Krzysztof Czarnecki, and Venkatesh Raman. 2015. SAT-based Analysis of Large Real-world Feature Models is Easy. In Proceedings of SPLC 2015, Douglas C. Schmidt (Ed.). 91–100.
[46]
Jinkun Lin, Shaowei Cai, Chuan Luo, Qingwei Lin, and Hongyu Zhang. 2019. Towards More Efficient Meta-heuristic Algorithms for Combinatorial Test Generation. In Proceedings of ESEC/FSE 2019. 212–222.
[47]
Jinkun Lin, Chuan Luo, Shaowei Cai, Kaile Su, Dan Hao, and Lu Zhang. 2015. TCA: An Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation. In Proceedings of ASE 2015. 494–505.
[48]
Chuan Luo, Shaowei Cai, Kaile Su, and Wenxuan Huang. 2017. CCEHC: An Efficient Local Search Algorithm for Weighted Partial Maximum Satisfiability. Artificial Intelligence, 243 (2017), 26–44.
[49]
Chuan Luo, Shaowei Cai, Kaile Su, and Wei Wu. 2015. Clause States Based Configuration Checking in Local Search for Satisfiability. IEEE Transactions on Cybernetics, 45, 5 (2015), 1014–1027.
[50]
Chuan Luo, Shaowei Cai, Wei Wu, Zhong Jie, and Kaile Su. 2015. CCLS: An Efficient Local Search Algorithm for Weighted Maximum Satisfiability. IEEE Transactions on Computers, 64, 7 (2015), 1830–1843.
[51]
Chuan Luo, Shaowei Cai, Wei Wu, and Kaile Su. 2013. Focused Random Walk with Configuration Checking and Break Minimum for Satisfiability. In Proceedings of CP 2013. 481–496.
[52]
Chuan Luo, Shaowei Cai, Wei Wu, and Kaile Su. 2014. Double Configuration Checking in Stochastic Local Search for Satisfiability. In Proceedings of AAAI 2014. 2703–2709.
[53]
Chuan Luo, Holger H. Hoos, and Shaowei Cai. 2020. PbO-CCSAT: Boosting Local Search for Satisfiability Using Programming by Optimisation. In Proceedings of PPSN 2020. 373–389.
[54]
Chuan Luo, Holger H. Hoos, Shaowei Cai, Qingwei Lin, Hongyu Zhang, and Dongmei Zhang. 2019. Local Search with Efficient Automatic Configuration for Minimum Vertex Cover. In Proceedings of IJCAI 2019. 1297–1304.
[55]
Chuan Luo, Jinkun Lin, Shaowei Cai, Xin Chen, Bing He, Bo Qiao, Pu Zhao, Qingwei Lin, Hongyu Zhang, Wei Wu, Saravanakumar Rajmohan, and Dongmei Zhang. 2021. AutoCCAG: An Automated Approach to Constrained Covering Array Generation. In Proceedings of ICSE 2021. 201–212.
[56]
Chuan Luo, Bo Qiao, Xin Chen, Pu Zhao, Randolph Yao, Hongyu Zhang, Wei Wu, Andrew Zhou, and Qingwei Lin. 2020. Intelligent Virtual Machine Provisioning in Cloud Computing. In Proceedings of IJCAI 2020. 1495–1502.
[57]
Chuan Luo, Bo Qiao, Wenqian Xing, Xin Chen, Pu Zhao, Chao Du, Randolph Yao, Hongyu Zhang, Wei Wu, Shaowei Cai, Bing He, Saravanakumar Rajmohan, and Qingwei Lin. 2021. Correlation-Aware Heuristic Search for Intelligent Virtual Machine Provisioning in Cloud Systems. In Proceedings of AAAI 2021. 12363–12372.
[58]
Chuan Luo, Kaile Su, and Shaowei Cai. 2012. Improving Local Search for Random 3-SAT Using Quantitative Configuration Checking. In Proceedings of ECAI 2012. 570–575.
[59]
Chuan Luo, Kaile Su, and Shaowei Cai. 2014. More efficient two-mode stochastic local search for random 3-satisfiability. Applied Intelligence, 41, 3 (2014), 665–680.
[60]
Chuan Luo, Binqi Sun, Bo Qiao, Junjie Chen, Hongyu Zhang, Jinkun Lin, Qingwei Lin, and Dongmei Zhang. 2021. LS-Sampling: An Effective Local Search based Sampling Approach for Achieving High t-wise Coverage. In Proceedings of ESEC/FSE 2021. 1081–1092.
[61]
Chuan Luo, Wenqian Xing, Shaowei Cai, and Chunming Hu. 2022. NuSC: An Effective Local Search Algorithm for Solving the Set Covering Problem. IEEE Transactions on Cybernetics.
[62]
Chuan Luo, Qiyuan Zhao, Shaowei Cai, Hongyu Zhang, and Chunming Hu. 2022. SamplingCA: Effective and Efficient Sampling-Based Pairwise Testing for Highly Configurable Software Systems. In Proceedings of ESEC/FSE 2022. 1185–1197.
[63]
Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, and Jianjun Zhao. 2019. DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems. In Proceedings of SANER 2019. 614–618.
[64]
Marcel Maehren, Philipp Nieting, Sven Hebrok, Robert Merget, Juraj Somorovsky, and Jörg Schwenk. 2022. TLS-Anvil: Adapting Combinatorial Testing for TLS Libraries. In Proceedings of USENIX Security 2022. 215–232.
[65]
Michalis Mavrovouniotis, Felipe Martins Müller, and Shengxiang Yang. 2017. Ant Colony Optimization With Local Search for Dynamic Traveling Salesman Problems. IEEE Transactions on Cybernetics, 47, 7 (2017), 1743–1756.
[66]
David A. McAllester, Bart Selman, and Henry A. Kautz. 1997. Evidence for Invariants in Local Search. In Proceedings of AAAI 1997. 321–326.
[67]
James D. McCaffrey. 2009. Generation of Pairwise Test Sets Using a Genetic Algorithm. In Proceedings of COMPSAC 2009. 626–631.
[68]
Flávio Medeiros, Christian Kästner, Márcio Ribeiro, Rohit Gheyi, and Sven Apel. 2016. A Comparison of 10 Sampling Algorithms for Configurable Systems. In Proceedings of ICSE 2016. 643–654.
[69]
Hanefi Mercan, Cemal Yilmaz, and Kamer Kaya. 2019. CHiP: A Configurable Hybrid Parallel Covering Array Constructor. IEEE Transactions on Software Engineering, 45, 12 (2019), 1270–1291.
[70]
Wil Michiels, Emile H. L. Aarts, and Jan H. M. Korst. 2007. Theoretical aspects of local search. Springer.
[71]
Changhai Nie and Hareton Leung. 2011. A Survey of Combinatorial Testing. Comput. Surveys, 43, 2 (2011), 11:1–11:29.
[72]
Jeho Oh, Paul Gazzillo, and Don S. Batory. 2019. t-wise Coverage by Uniform Sampling. In Proceedings of SPLC 2019. 15:1–15:4.
[73]
Sebastian Oster, Florian Markert, and Philipp Ritter. 2010. Automated Incremental Pairwise Testing of Software Product Lines. In Proceedings of SPLC 2010. 196–210.
[74]
Mingyu Park, Hoon Jang, Taejoon Byun, and Yunja Choi. 2020. Property-based testing for LG home appliances using accelerated software-in-the-loop simulation. In Proceedings of ICSE-SEIP 2020. 120–129.
[75]
Justyna Petke, Myra B. Cohen, Mark Harman, and Shin Yoo. 2015. Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection. IEEE Transactions on Software Engineering, 41, 9 (2015), 901–924.
[76]
Justyna Petke, Shin Yoo, Myra B. Cohen, and Mark Harman. 2013. Efficiency and early fault detection with lower and higher strength combinatorial interaction testing. In Proceedings of ESEC/FSE 2013. 26–36.
[77]
Tobias Pett, Thomas Thüm, Tobias Runge, Sebastian Krieter, Malte Lochau, and Ina Schaefer. 2019. Product Sampling for Product Lines: The Scalability Challenge. In Proceedings of SPLC 2019. 14:1–14:6.
[78]
Quentin Plazar, Mathieu Acher, Gilles Perrouin, Xavier Devroey, and Maxime Cordy. 2019. Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet? In Proceedings of ICST 2019. 240–251.
[79]
Bo Qiao, Fangkai Yang, Chuan Luo, Yanan Wang, Johnny Li, Qingwei Lin, Hongyu Zhang, Mohit Datta, Andrew Zhou, Thomas Moscibroda, Saravanakumar Rajmohan, and Dongmei Zhang. 2021. Intelligent Container Reallocation at Microsoft 365. In Proceedings of ESEC/FSE 2021. 1438–1443.
[80]
Federica Sarro, Mark Harman, Yue Jia, and Yuanyuan Zhang. 2018. Customer Rating Reactions Can Be Predicted Purely using App Features. In Proceedings of RE 2018. 76–87.
[81]
Jialong Shi, Qingfu Zhang, and Jianyong Sun. 2020. PPLS/D: Parallel Pareto Local Search Based on Decomposition. IEEE Transactions on Cybernetics, 50, 3 (2020), 1060–1071.
[82]
Jing Sun, Hongyu Zhang, Yuan-Fang Li, and Hai H. Wang. 2005. Formal Semantics and Verification for Feature Modeling. In Proceedings of ICECCS 2005. 303–312.
[83]
Kuo-Chung Tai and Yu Lei. 2002. A Test Generation Strategy for Pairwise Testing. IEEE Transactions on Software Engineering, 28, 1 (2002), 109–111.
[84]
Thomas Thüm, Sven Apel, Christian Kästner, Ina Schaefer, and Gunter Saake. 2014. A Classification and Survey of Analysis Strategies for Software Product Lines. ACM Computing Surveys, 47, 1 (2014), 6:1–6:45.
[85]
Yu-Wen Tung and Wafa S. Aldiwan. 2000. Automating Test Case Generation for the New Generation Mission Software System. In Proceedings of IEEE Aerospace Conference 2000. 431–437.
[86]
András Vargha and Harold D. Delaney. 2000. A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25, 2 (2000), 101–132.
[87]
Xinyi Wang, Paolo Arcaini, Tao Yue, and Shaukat Ali. 2021. Application of Combinatorial Testing to Quantum Programs. In Proceedings of QRS 2021. 179–188.
[88]
Ziyuan Wang, Changhai Nie, and Baowen Xu. 2007. Generating Combinatorial Test Suite for Interaction Relationship. In Proceedings of SOQUA 2007. 55–61.
[89]
Huayao Wu, Lixin Xu, Xintao Niu, and Changhai Nie. 2022. Combinatorial testing of restful apis. In Proceedings of ICSE 2022. 426–437.
[90]
Yi Xiang, Han Huang, Miqing Li, Sizhe Li, and Xiaowei Yang. 2022. Looking For Novelty in Search-Based Software Product Line Testing. IEEE Transactions on Software Engineering, 48, 7 (2022), 2317–2338.
[91]
Akihisa Yamada, Armin Biere, Cyrille Artho, Takashi Kitamura, and Eun-Hye Choi. 2016. Greedy Combinatorial Test Case Generation using Unsatisfiable Cores. In Proceedings of ASE 2016. 614–624.
[92]
Akihisa Yamada, Takashi Kitamura, Cyrille Artho, Eun-Hye Choi, Yutaka Oiwa, and Armin Biere. 2015. Optimization of Combinatorial Testing by Incremental SAT Solving. In Proceedings of ICST 2015. 1–10.
[93]
Cemal Yilmaz, Myra B. Cohen, and Adam A. Porter. 2006. Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces. IEEE Transactions on Software Engineering, 32, 1 (2006), 20–34.
[94]
Cemal Yilmaz, Sandro Fouché, Myra B. Cohen, Adam A. Porter, Gülsen Demiröz, and Ugur Koc. 2014. Moving Forward with Combinatorial Interaction Testing. Computer, 47, 2 (2014), 37–45.
[95]
Linbin Yu, Yu Lei, Mehra Nouroz Borazjany, Raghu Kacker, and D. Richard Kuhn. 2013. An Efficient Algorithm for Constraint Handling in Combinatorial Test Generation. In Proceedings of ICST 2013. 242–251.
[96]
Jian Zhang, Zhiqiang Zhang, and Feifei Ma. 2014. Automatic Generation of Combinatorial Test Data. Springer. isbn:978-3-662-43428-4
[97]
Xindi Zhang, Bohan Li, Shaowei Cai, and Yiyuan Wang. 2021. Efficient Local Search based on Dynamic Connectivity Maintenance for Minimum Connected Dominating Set. Journal of Artificial Intelligence Research, 71 (2021), 89–119.
[98]
Zhiqiang Zhang, Jun Yan, Yong Zhao, and Jian Zhang. 2014. Generating Combinatorial Test Suite using Combinatorial Optimization. Journal of Systems and Software, 98 (2014), 191–207.
Index Terms
- CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays
Recommendations
SamplingCA: effective and efficient sampling-based pairwise testing for highly configurable software systems
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software EngineeringCombinatorial interaction testing (CIT) is an effective paradigm for testing highly configurable systems, and its goal is to generate a t-wise covering array (CA) as a test suite, where t is the strength of testing. It is recognized that pairwise ...
LS-sampling: an effective local search based sampling approach for achieving high t-wise coverage
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringThere has been a rapidly increasing demand for developing highly configurable software systems, which urgently calls for effective testing methods. In practice, t-wise coverage has been widely recognized as a useful metric to evaluate the quality of a ...
A Test Generation Strategy for Pairwise Testing
Pairwise testing is a specification-based testing criterion, which requires that for each pair of input parameters of a system, every combination of valid values of these two parameters be covered by at least one test case. In this paper, we propose a ...
Comments
Information & Contributors
Information
Published In
November 2023
2215 pages
ISBN:9798400703270
DOI:10.1145/3611643
- General Chair:
- Satish Chandra,
- Program Chairs:
- Kelly Blincoe,
- Paolo Tonella
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 30 November 2023
Check for updates
Badges
Author Tags
Qualifiers
- Research-article
Funding Sources
- National Key Research and Development Program of China
- National Natural Science Foundation of China
- CCF-Huawei Populus Grove Fund
- Open Project of Xiangjiang Laboratory
- Natural Science Foundation of Changsha
Conference
ESEC/FSE '23
Sponsor:
ESEC/FSE '23: 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
December 3 - 9, 2023
CA, San Francisco, USA
Acceptance Rates
Overall Acceptance Rate 112 of 543 submissions, 21%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 235Total Downloads
- Downloads (Last 12 months)125
- Downloads (Last 6 weeks)3
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in