research-article

CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays

Authors:
Qiyuan Zhao

Beihang University, Beijing, China

Beihang University, Beijing, China
View Profile

,
Chuan Luo

Beihang University, Beijing, China

Beihang University, Beijing, China
View Profile

,
Shaowei Cai

Institute of Software, Chinese Academy of Sciences, Beijing, China

Institute of Software, Chinese Academy of Sciences, Beijing, China
View Profile

,
Wei Wu

Central South University, Changsha, China / Xiangjiang Laboratory, Changsha, China

Central South University, Changsha, China / Xiangjiang Laboratory, Changsha, China
View Profile

,
Jinkun Lin

SeedMath Technology Limited, Beijing, China

SeedMath Technology Limited, Beijing, China
View Profile

,
Hongyu Zhang

Chongqing University, Chongqing, China

Chongqing University, Chongqing, China
View Profile

,
Chunming Hu

Beihang University, Beijing, China

Beihang University, Beijing, China
View Profile

ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software EngineeringNovember 2023Pages 81–93https://doi.org/10.1145/3611643.3616284

Published:30 November 2023Publication History

ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Pages 81–93

ABSTRACT

The increasing demand for software customization has led to the development of highly configurable systems. Combinatorial interaction testing (CIT) is an effective method for testing these types of systems. The ultimate goal of CIT is to generate a test suite of acceptable size, called a t-wise covering array (CA), where t is the testing strength. Pairwise testing (i.e., CIT with t=2) is recognized to be the most widely-used CIT technique and has strong fault detection capability. In pairwise testing, the most important problem is pairwise CA generation (PCAG), which is to generate a pairwise CA (PCA) of minimum size. However, existing state-of-the-art PCAG algorithms suffer from the severe scalability challenge; that is, they cannot tackle large-scale PCAG instances effectively, resulting in PCAs of large sizes. To alleviate this challenge, in this paper we propose CAmpactor, a novel and effective local search algorithm for compacting given PCAs into smaller sizes. Extensive experiments on a large number of real-world, public PCAG instances show that the sizes of CAmpactor's generated PCAs are around 45% smaller than the sizes of PCAs constructed by existing state-of-the-art PCAG algorithms, indicating its superiority. Also, our evaluation confirms the generality of CAmpactor, since CAmpactor can reduce the sizes of PCAs generated by a variety of PCAG algorithms.

References

Carlos Ansótegui, Felip Manyà, Jesus Ojeda, Josep M. Salvia, and Eduard Torres. 2022. Incomplete MaxSAT approaches for combinatorial testing. Journal of Heuristics, 28, 4 (2022), 377–431. Google ScholarDigital Library
Sven Apel, Don S. Batory, Christian Kästner, and Gunter Saake. 2013. Feature-Oriented Software Product Lines - Concepts and Implementation. Springer. Google Scholar
Adrian Balint and Andreas Fröhlich. 2010. Improving Stochastic Local Search for SAT with a New Probability Distribution. In Proceedings of SAT 2010. Springer, 10–15. Google ScholarDigital Library
Mutsunori Banbara, Haruki Matsunaka, Naoyuki Tamura, and Katsumi Inoue. 2010. Generating Combinatorial Test Cases by Efficient SAT Encodings Suitable for CDCL SAT Solvers. In Proceedings of LPAR 2010. 112–126. Google ScholarDigital Library
Eduard Baranov, Axel Legay, and Kuldeep S. Meel. 2020. Baital: An Adaptive Weighted Sampling Approach for Improved t-wise Coverage. In Proceedings of ESEC/FSE 2020. 1114–1126. Google ScholarDigital Library
Don S. Batory. 2005. Feature Models, Grammars, and Propositional Formulas. In Proceedings of SPLC 2005. 7–20. Google ScholarDigital Library
2009. Handbook of Satisfiability, Armin Biere, Marijn Heule, Hans van Maaren, and Toby Walsh (Eds.) (Frontiers in Artificial Intelligence and Applications, Vol. 185). IOS Press. isbn:978-1-58603-929-5 Google Scholar
Renée C. Bryce and Charles J. Colbourn. 2007. The Density Algorithm for Pairwise Interaction Testing. Software Testing, Verification and Reliability, 17, 3 (2007), 159–182. Google ScholarDigital Library
Renée C. Bryce and Charles J. Colbourn. 2009. A Density-based Greedy Algorithm for Higher Strength Covering Arrays. Software Testing, Verification and Reliability, 19, 1 (2009), 37–53. Google ScholarDigital Library
Renée C. Bryce, Charles J. Colbourn, and Myra B. Cohen. 2005. A Framework of Greedy Methods for Constructing Interaction Test Suites. In Proceedings of ICSE 2005. 146–155. Google Scholar
Shaowei Cai, Jinkun Lin, and Chuan Luo. 2017. Finding A Small Vertex Cover in Massive Sparse Graphs: Construct, Local Search, and Preprocess. Journal of Artificial Intelligence Research, 59 (2017), 463–494. Google ScholarDigital Library
Shaowei Cai and Kaile Su. 2013. Local search for Boolean Satisfiability with configuration checking and subscore. Artificial Intelligence, 204 (2013), 75–98. Google ScholarDigital Library
Xinye Cai, Haoran Sun, Qingfu Zhang, and Yuhua Huang. 2019. A Grid Weighted Sum Pareto Local Search for Combinatorial Multi and Many-Objective Optimization. IEEE Transactions on Cybernetics, 49, 9 (2019), 3586–3598. Google ScholarCross Ref
Yanshan Chen, Ziyuan Wang, Dong Wang, Chunrong Fang, and Zhenyu Chen. 2019. Variable Strength Combinatorial Testing for Deep Neural Networks. In Proceedings of ICST Workshops 2019. 281–284. Google ScholarCross Ref
Yi Chu, Shaowei Cai, and Chuan Luo. 2023. NuWLS: Improving Local Search for (Weighted) Partial MaxSAT by New Weighting Techniques. In Proceedings of AAAI 2023. 3915–3923. Google ScholarDigital Library
David M. Cohen, Siddhartha R. Dalal, Michael L. Fredman, and Gardner C. Patton. 1997. The AETG System: An Approach to Testing Based on Combinatorial Design. IEEE Transactions on Software Engineering, 23, 7 (1997), 437–444. Google ScholarDigital Library
Myra B. Cohen, Charles J. Colbourn, and Alan C. H. Ling. 2003. Augmenting Simulated Annealing to Build Interaction Test Suites. In Proceedings of ISSRE 2003. 394–405. Google Scholar
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, and Charles J. Colbourn. 2003. Constructing Test Suites for Interaction Testing. In Proceedings ICSE 2003. 38–48. Google Scholar
Myra B. Cohen, Peter B. Gibbons, Warwick B. Mugridge, Charles J. Colbourn, and James S. Collofello. 2003. A Variable Strength Interaction Testing of Components. In Proceedings of COMPAC 2003. 413–418. Google Scholar
W. J. Conover. 1999. Practical Nonparametric Statistics. Conover. isbn:Conover Google Scholar
Hang Dong, Boshi Wang, Bo Qiao, Wenqian Xing, Chuan Luo, Si Qin, Qingwei Lin, Dongmei Zhang, Gurpreet Virdi, and Thomas Moscibroda. 2021. Predictive Job Scheduling under Uncertain Constraints in Cloud Computing. In Proceedings of IJCAI 2021. 3627–3634. Google ScholarCross Ref
Niklas Eén and Niklas Sörensson. 2003. An Extensible SAT-solver. In Proceedings of SAT 2003. 502–518. Google Scholar
Brady J. Garvin, Myra B. Cohen, and Matthew B. Dwyer. 2009. An Improved Meta-Heuristic Search for Constrained Interaction Testing. In Proceedings of International Symposium on Search Based Software Engineering 2009. 13–22. Google Scholar
Brady J. Garvin, Myra B. Cohen, and Matthew B. Dwyer. 2011. Evaluating Improvements to a Meta-heuristic Search for Constrained Interaction Testing. Empirical Software Engineering, 16, 1 (2011), 61–102. Google ScholarDigital Library
Syed A. Ghazi and Moataz A. Ahmed. 2003. Pair-wise Test Coverage using Genetic Algorithms. In Proceedings of CEC 2003. 1420–1424. Google Scholar
Harrison Goldstein, John Hughes, Leonidas Lampropoulos, and Benjamin C. Pierce. 2021. Do Judge a Test by its Cover - Combining Combinatorial and Property-Based Testing. In Proceedings of ESOP 2021. 264–291. Google Scholar
Loreto Gonzalez-Hernandez, Nelson Rangel-Valdez, and Jose Torres-Jimenez. 2010. Construction of Mixed Covering Arrays of Variable Strength Using a Tabu Search Approach. In Proceedings of COCOA 2010. 51–64. Google ScholarCross Ref
Jiazhen Gu, Chuan Luo, Si Qin, Bo Qiao, Qingwei Lin, Hongyu Zhang, Ze Li, Yingnong Dang, Shaowei Cai, Wei Wu, Yangfan Zhou, Murali Chintalapati, and Dongmei Zhang. 2020. Efficient Incident Identification from Multi-dimensional Issue Reports via Meta-heuristic Search. In Proceedings of ESEC/FSE 2020. 292–303. Google ScholarDigital Library
Christopher Henard, Mike Papadakis, Gilles Perrouin, Jacques Klein, Patrick Heymans, and Yves Le Traon. 2014. Bypassing the Combinatorial Explosion: Using Similarity to Generate and Prioritize T-Wise Test Configurations for Software Product Lines. IEEE Transactions on Software Engineering, 40, 7 (2014), 650–670. Google ScholarDigital Library
Aymeric Hervieu, Dusica Marijan, Arnaud Gotlieb, and Benoit Baudry. 2016. Practical Minimization of Pairwise-covering Test Configurations using Constraint Programming. Information and Software Technology, 71 (2016), 129–146. Google ScholarDigital Library
Brahim Hnich, Steven David Prestwich, Evgeny Selensky, and Barbara M. Smith. 2006. Constraint Models for the Covering Test Problem. Constraints, 11, 2-3 (2006), 199–219. Google ScholarDigital Library
Holger H. Hoos. 2002. An Adaptive Noise Mechanism for WalkSAT. In Proceedings of AAAI 2002. 655–660. Google Scholar
Holger H. Hoos and Thomas Stützle. 2004. Stochastic Local Search: Foundations & Applications. Elsevier / Morgan Kaufmann. isbn:1-55860-872-9 Google Scholar
Yue Jia, Myra B. Cohen, Mark Harman, and Justyna Petke. 2015. Learning Combinatorial Interaction Test Generation Strategies Using Hyperheuristic Search. In Proceedings of ICSE 2015. 540–550. Google ScholarDigital Library
Christian Kaltenecker, Alexander Grebhahn, Norbert Siegmund, Jianmei Guo, and Sven Apel. 2019. Distance-based Sampling of Software Configuration Spaces. In Proceedings of ICSE 2019. 1084–1094. Google ScholarDigital Library
Alexander Knüppel, Thomas Thüm, Stephan Mennicke, Jens Meinicke, and Ina Schaefer. 2017. Is There a Mismatch between Real-world Feature Models and Product-line Research? In Proceedings of ESEC/FSE 2017. 291–302. Google ScholarDigital Library
Sebastian Krieter, Thomas Thüm, Sandro Schulze, Gunter Saake, and Thomas Leich. 2020. YASA: yet another sampling algorithm. In Proceedings of VaMoS 2020. 4:1–4:10. Google ScholarDigital Library
D. Richard Kuhn, Raghu N. Kacker, and Yu Lei. 2013. Introduction to Combinatorial Testing. CRC press. isbn:978-1-466-55229-6 Google Scholar
D. Richard Kuhn, Dolores R. Wallace, and Albert M. Gallo. 2004. Software Fault Interactions and Implications for Software Testing. IEEE Transactions on Software Engineering, 30, 6 (2004), 418–421. Google ScholarDigital Library
Rick Kuhn, Raghu N. Kacker, Jeff Yu Lei, and Dimitris E. Simos. 2020. Input Space Coverage Matters. Computer, 53, 1 (2020), 37–44. Google ScholarCross Ref
Yu Lei, Raghu Kacker, D. Richard Kuhn, Vadim Okun, and James Lawrence. 2007. IPOG: A General Strategy for T-Way Software Testing. In Proceedings of ECBS 2007. 549–556. Google ScholarDigital Library
Yu Lei, Raghu Kacker, D. Richard Kuhn, Vadim Okun, and James Lawrence. 2008. IPOG/IPOG-D: Efficient Test Generation for Multi-way Combinatorial Testing. Software Testing, Verification and Reliability, 18, 3 (2008), 125–148. Google ScholarDigital Library
Yu Lei and Kuo-Chung Tai. 1998. In-Parameter-Order: A Test Generation Strategy for Pairwise Testing. In Proceedings of HASE 1998. 254–261. Google ScholarDigital Library
Chu Min Li and Wenqi Huang. 2005. Diversification and Determinism in Local Search for Satisfiability. In Proceedings of SAT 2005. 158–172. Google ScholarDigital Library
Jia Hui Liang, Vijay Ganesh, Krzysztof Czarnecki, and Venkatesh Raman. 2015. SAT-based Analysis of Large Real-world Feature Models is Easy. In Proceedings of SPLC 2015, Douglas C. Schmidt (Ed.). 91–100. Google ScholarDigital Library
Jinkun Lin, Shaowei Cai, Chuan Luo, Qingwei Lin, and Hongyu Zhang. 2019. Towards More Efficient Meta-heuristic Algorithms for Combinatorial Test Generation. In Proceedings of ESEC/FSE 2019. 212–222. Google ScholarDigital Library
Jinkun Lin, Chuan Luo, Shaowei Cai, Kaile Su, Dan Hao, and Lu Zhang. 2015. TCA: An Efficient Two-Mode Meta-Heuristic Algorithm for Combinatorial Test Generation. In Proceedings of ASE 2015. 494–505. Google ScholarDigital Library
Chuan Luo, Shaowei Cai, Kaile Su, and Wenxuan Huang. 2017. CCEHC: An Efficient Local Search Algorithm for Weighted Partial Maximum Satisfiability. Artificial Intelligence, 243 (2017), 26–44. Google ScholarDigital Library
Chuan Luo, Shaowei Cai, Kaile Su, and Wei Wu. 2015. Clause States Based Configuration Checking in Local Search for Satisfiability. IEEE Transactions on Cybernetics, 45, 5 (2015), 1014–1027. Google ScholarCross Ref
Chuan Luo, Shaowei Cai, Wei Wu, Zhong Jie, and Kaile Su. 2015. CCLS: An Efficient Local Search Algorithm for Weighted Maximum Satisfiability. IEEE Transactions on Computers, 64, 7 (2015), 1830–1843. Google ScholarDigital Library
Chuan Luo, Shaowei Cai, Wei Wu, and Kaile Su. 2013. Focused Random Walk with Configuration Checking and Break Minimum for Satisfiability. In Proceedings of CP 2013. 481–496. Google ScholarCross Ref
Chuan Luo, Shaowei Cai, Wei Wu, and Kaile Su. 2014. Double Configuration Checking in Stochastic Local Search for Satisfiability. In Proceedings of AAAI 2014. 2703–2709. Google ScholarCross Ref
Chuan Luo, Holger H. Hoos, and Shaowei Cai. 2020. PbO-CCSAT: Boosting Local Search for Satisfiability Using Programming by Optimisation. In Proceedings of PPSN 2020. 373–389. Google ScholarDigital Library
Chuan Luo, Holger H. Hoos, Shaowei Cai, Qingwei Lin, Hongyu Zhang, and Dongmei Zhang. 2019. Local Search with Efficient Automatic Configuration for Minimum Vertex Cover. In Proceedings of IJCAI 2019. 1297–1304. Google ScholarCross Ref
Chuan Luo, Jinkun Lin, Shaowei Cai, Xin Chen, Bing He, Bo Qiao, Pu Zhao, Qingwei Lin, Hongyu Zhang, Wei Wu, Saravanakumar Rajmohan, and Dongmei Zhang. 2021. AutoCCAG: An Automated Approach to Constrained Covering Array Generation. In Proceedings of ICSE 2021. 201–212. Google ScholarDigital Library
Chuan Luo, Bo Qiao, Xin Chen, Pu Zhao, Randolph Yao, Hongyu Zhang, Wei Wu, Andrew Zhou, and Qingwei Lin. 2020. Intelligent Virtual Machine Provisioning in Cloud Computing. In Proceedings of IJCAI 2020. 1495–1502. Google ScholarCross Ref
Chuan Luo, Bo Qiao, Wenqian Xing, Xin Chen, Pu Zhao, Chao Du, Randolph Yao, Hongyu Zhang, Wei Wu, Shaowei Cai, Bing He, Saravanakumar Rajmohan, and Qingwei Lin. 2021. Correlation-Aware Heuristic Search for Intelligent Virtual Machine Provisioning in Cloud Systems. In Proceedings of AAAI 2021. 12363–12372. Google ScholarCross Ref
Chuan Luo, Kaile Su, and Shaowei Cai. 2012. Improving Local Search for Random 3-SAT Using Quantitative Configuration Checking. In Proceedings of ECAI 2012. 570–575. Google Scholar
Chuan Luo, Kaile Su, and Shaowei Cai. 2014. More efficient two-mode stochastic local search for random 3-satisfiability. Applied Intelligence, 41, 3 (2014), 665–680. Google ScholarDigital Library
Chuan Luo, Binqi Sun, Bo Qiao, Junjie Chen, Hongyu Zhang, Jinkun Lin, Qingwei Lin, and Dongmei Zhang. 2021. LS-Sampling: An Effective Local Search based Sampling Approach for Achieving High t-wise Coverage. In Proceedings of ESEC/FSE 2021. 1081–1092. Google ScholarDigital Library
Chuan Luo, Wenqian Xing, Shaowei Cai, and Chunming Hu. 2022. NuSC: An Effective Local Search Algorithm for Solving the Set Covering Problem. IEEE Transactions on Cybernetics. Google ScholarCross Ref
Chuan Luo, Qiyuan Zhao, Shaowei Cai, Hongyu Zhang, and Chunming Hu. 2022. SamplingCA: Effective and Efficient Sampling-Based Pairwise Testing for Highly Configurable Software Systems. In Proceedings of ESEC/FSE 2022. 1185–1197. Google Scholar
Lei Ma, Felix Juefei-Xu, Minhui Xue, Bo Li, Li Li, Yang Liu, and Jianjun Zhao. 2019. DeepCT: Tomographic Combinatorial Testing for Deep Learning Systems. In Proceedings of SANER 2019. 614–618. Google ScholarCross Ref
Marcel Maehren, Philipp Nieting, Sven Hebrok, Robert Merget, Juraj Somorovsky, and Jörg Schwenk. 2022. TLS-Anvil: Adapting Combinatorial Testing for TLS Libraries. In Proceedings of USENIX Security 2022. 215–232. Google Scholar
Michalis Mavrovouniotis, Felipe Martins Müller, and Shengxiang Yang. 2017. Ant Colony Optimization With Local Search for Dynamic Traveling Salesman Problems. IEEE Transactions on Cybernetics, 47, 7 (2017), 1743–1756. Google ScholarCross Ref
David A. McAllester, Bart Selman, and Henry A. Kautz. 1997. Evidence for Invariants in Local Search. In Proceedings of AAAI 1997. 321–326. Google Scholar
James D. McCaffrey. 2009. Generation of Pairwise Test Sets Using a Genetic Algorithm. In Proceedings of COMPSAC 2009. 626–631. Google ScholarDigital Library
Flávio Medeiros, Christian Kästner, Márcio Ribeiro, Rohit Gheyi, and Sven Apel. 2016. A Comparison of 10 Sampling Algorithms for Configurable Systems. In Proceedings of ICSE 2016. 643–654. Google ScholarDigital Library
Hanefi Mercan, Cemal Yilmaz, and Kamer Kaya. 2019. CHiP: A Configurable Hybrid Parallel Covering Array Constructor. IEEE Transactions on Software Engineering, 45, 12 (2019), 1270–1291. Google ScholarCross Ref
Wil Michiels, Emile H. L. Aarts, and Jan H. M. Korst. 2007. Theoretical aspects of local search. Springer. Google Scholar
Changhai Nie and Hareton Leung. 2011. A Survey of Combinatorial Testing. Comput. Surveys, 43, 2 (2011), 11:1–11:29. Google Scholar
Jeho Oh, Paul Gazzillo, and Don S. Batory. 2019. t-wise Coverage by Uniform Sampling. In Proceedings of SPLC 2019. 15:1–15:4. Google ScholarDigital Library
Sebastian Oster, Florian Markert, and Philipp Ritter. 2010. Automated Incremental Pairwise Testing of Software Product Lines. In Proceedings of SPLC 2010. 196–210. Google ScholarCross Ref
Mingyu Park, Hoon Jang, Taejoon Byun, and Yunja Choi. 2020. Property-based testing for LG home appliances using accelerated software-in-the-loop simulation. In Proceedings of ICSE-SEIP 2020. 120–129. Google Scholar
Justyna Petke, Myra B. Cohen, Mark Harman, and Shin Yoo. 2015. Practical Combinatorial Interaction Testing: Empirical Findings on Efficiency and Early Fault Detection. IEEE Transactions on Software Engineering, 41, 9 (2015), 901–924. Google ScholarDigital Library
Justyna Petke, Shin Yoo, Myra B. Cohen, and Mark Harman. 2013. Efficiency and early fault detection with lower and higher strength combinatorial interaction testing. In Proceedings of ESEC/FSE 2013. 26–36. Google ScholarDigital Library
Tobias Pett, Thomas Thüm, Tobias Runge, Sebastian Krieter, Malte Lochau, and Ina Schaefer. 2019. Product Sampling for Product Lines: The Scalability Challenge. In Proceedings of SPLC 2019. 14:1–14:6. Google ScholarDigital Library
Quentin Plazar, Mathieu Acher, Gilles Perrouin, Xavier Devroey, and Maxime Cordy. 2019. Uniform Sampling of SAT Solutions for Configurable Systems: Are We There Yet? In Proceedings of ICST 2019. 240–251. Google ScholarCross Ref
Bo Qiao, Fangkai Yang, Chuan Luo, Yanan Wang, Johnny Li, Qingwei Lin, Hongyu Zhang, Mohit Datta, Andrew Zhou, Thomas Moscibroda, Saravanakumar Rajmohan, and Dongmei Zhang. 2021. Intelligent Container Reallocation at Microsoft 365. In Proceedings of ESEC/FSE 2021. 1438–1443. Google ScholarDigital Library
Federica Sarro, Mark Harman, Yue Jia, and Yuanyuan Zhang. 2018. Customer Rating Reactions Can Be Predicted Purely using App Features. In Proceedings of RE 2018. 76–87. Google ScholarCross Ref
Jialong Shi, Qingfu Zhang, and Jianyong Sun. 2020. PPLS/D: Parallel Pareto Local Search Based on Decomposition. IEEE Transactions on Cybernetics, 50, 3 (2020), 1060–1071. Google ScholarCross Ref
Jing Sun, Hongyu Zhang, Yuan-Fang Li, and Hai H. Wang. 2005. Formal Semantics and Verification for Feature Modeling. In Proceedings of ICECCS 2005. 303–312. Google Scholar
Kuo-Chung Tai and Yu Lei. 2002. A Test Generation Strategy for Pairwise Testing. IEEE Transactions on Software Engineering, 28, 1 (2002), 109–111. Google ScholarDigital Library
Thomas Thüm, Sven Apel, Christian Kästner, Ina Schaefer, and Gunter Saake. 2014. A Classification and Survey of Analysis Strategies for Software Product Lines. ACM Computing Surveys, 47, 1 (2014), 6:1–6:45. Google Scholar
Yu-Wen Tung and Wafa S. Aldiwan. 2000. Automating Test Case Generation for the New Generation Mission Software System. In Proceedings of IEEE Aerospace Conference 2000. 431–437. Google Scholar
András Vargha and Harold D. Delaney. 2000. A Critique and Improvement of the CL Common Language Effect Size Statistics of McGraw and Wong. Journal of Educational and Behavioral Statistics, 25, 2 (2000), 101–132. Google Scholar
Xinyi Wang, Paolo Arcaini, Tao Yue, and Shaukat Ali. 2021. Application of Combinatorial Testing to Quantum Programs. In Proceedings of QRS 2021. 179–188. Google ScholarCross Ref
Ziyuan Wang, Changhai Nie, and Baowen Xu. 2007. Generating Combinatorial Test Suite for Interaction Relationship. In Proceedings of SOQUA 2007. 55–61. Google Scholar
Huayao Wu, Lixin Xu, Xintao Niu, and Changhai Nie. 2022. Combinatorial testing of restful apis. In Proceedings of ICSE 2022. 426–437. Google ScholarDigital Library
Yi Xiang, Han Huang, Miqing Li, Sizhe Li, and Xiaowei Yang. 2022. Looking For Novelty in Search-Based Software Product Line Testing. IEEE Transactions on Software Engineering, 48, 7 (2022), 2317–2338. Google ScholarCross Ref
Akihisa Yamada, Armin Biere, Cyrille Artho, Takashi Kitamura, and Eun-Hye Choi. 2016. Greedy Combinatorial Test Case Generation using Unsatisfiable Cores. In Proceedings of ASE 2016. 614–624. Google ScholarDigital Library
Akihisa Yamada, Takashi Kitamura, Cyrille Artho, Eun-Hye Choi, Yutaka Oiwa, and Armin Biere. 2015. Optimization of Combinatorial Testing by Incremental SAT Solving. In Proceedings of ICST 2015. 1–10. Google ScholarCross Ref
Cemal Yilmaz, Myra B. Cohen, and Adam A. Porter. 2006. Covering Arrays for Efficient Fault Characterization in Complex Configuration Spaces. IEEE Transactions on Software Engineering, 32, 1 (2006), 20–34. Google ScholarDigital Library
Cemal Yilmaz, Sandro Fouché, Myra B. Cohen, Adam A. Porter, Gülsen Demiröz, and Ugur Koc. 2014. Moving Forward with Combinatorial Interaction Testing. Computer, 47, 2 (2014), 37–45. Google ScholarDigital Library
Linbin Yu, Yu Lei, Mehra Nouroz Borazjany, Raghu Kacker, and D. Richard Kuhn. 2013. An Efficient Algorithm for Constraint Handling in Combinatorial Test Generation. In Proceedings of ICST 2013. 242–251. Google Scholar
Jian Zhang, Zhiqiang Zhang, and Feifei Ma. 2014. Automatic Generation of Combinatorial Test Data. Springer. isbn:978-3-662-43428-4 Google Scholar
Xindi Zhang, Bohan Li, Shaowei Cai, and Yiyuan Wang. 2021. Efficient Local Search based on Dynamic Connectivity Maintenance for Minimum Connected Dominating Set. Journal of Artificial Intelligence Research, 71 (2021), 89–119. Google ScholarDigital Library
Zhiqiang Zhang, Jun Yan, Yong Zhao, and Jian Zhang. 2014. Generating Combinatorial Test Suite using Combinatorial Optimization. Journal of Systems and Software, 98 (2014), 191–207. Google ScholarDigital Library

Index Terms

CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays
1. Software and its engineering
  1. Software creation and management
    1. Search-based software engineering
    2. Software verification and validation
      1. Software defect analysis
        Software testing and debugging
2. Theory of computation
  1. Design and analysis of algorithms
    1. Mathematical optimization
      1. Discrete optimization
        Optimization with randomized search heuristics
        Randomized local search

Recommendations

Generating Pairwise Covering Arrays for Highly Configurable Software Systems
SPLC '23: Proceedings of the 27th ACM International Systems and Software Product Line Conference - Volume A

Highly configurable software systems play crucial roles in real-world applications, which urgently calls for useful testing methods. Combinatorial interaction testing (CIT) is an effective methodology for detecting those faults that are triggered by the ...
Read More
SamplingCA: effective and efficient sampling-based pairwise testing for highly configurable software systems
ESEC/FSE 2022: Proceedings of the 30th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

Combinatorial interaction testing (CIT) is an effective paradigm for testing highly configurable systems, and its goal is to generate a t-wise covering array (CA) as a test suite, where t is the strength of testing. It is recognized that pairwise ...
Read More
LS-sampling: an effective local search based sampling approach for achieving high t-wise coverage
ESEC/FSE 2021: Proceedings of the 29th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering

There has been a rapidly increasing demand for developing highly configurable software systems, which urgently calls for effective testing methods. In practice, t-wise coverage has been widely recognized as a useful metric to evaluate the quality of a ...
Read More

Comments

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Published in
ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering
November 2023
2215 pages
ISBN:9798400703270
DOI:10.1145/3611643
General Chair:
Satish Chandra
Google, USA
,
Program Chairs:
Kelly Blincoe
University of Auckland, New Zealand
,
Paolo Tonella
USI Lugano, Switzerland
Copyright © 2023 ACM
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
In-Cooperation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
- Published: 30 November 2023
Permissions
Request permissions about this article.
Request Permissions

Check for updates
Badges
- Artifacts Available / v1.1
- Artifacts Evaluated & Reusable / v1.1
Author Tags
Covering Array
Local Search
Optimization
Pairwise Testing
Qualifiers
- research-article
Conference

Acceptance Rates
Overall Acceptance Rate112of543submissions,21%
Funding Sources
Other Metrics
View Article Metrics

Article Metrics
- 0
  Total Citations
  View Citations
- 131
  Total Downloads
- Downloads (Last 12 months)131
- Downloads (Last 6 weeks)14
Other Metrics
View Author Metrics
Cited By
This publication has not been cited yet

PDF Format

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

CAmpactor: A Novel and Effective Local Search Algorithm for Optimizing Pairwise Covering Arrays

ESEC/FSE 2023: Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering

ABSTRACT

References

Cited By

Index Terms

Recommendations

Generating Pairwise Covering Arrays for Highly Configurable Software Systems

SamplingCA: effective and efficient sampling-based pairwise testing for highly configurable software systems

LS-sampling: an effective local search based sampling approach for achieving high t-wise coverage