Skip to main content
SpringerLink
Log in

A tolerance-based memetic algorithm for constrained covering array generation

  • Regular research paper
  • Published:
Memetic Computing Aims and scope Submit manuscript

Abstract

The research focus in the field of combinatorial testing has shifted from unconstrained covering array generation (CAG) to constrained covering array generation (CCAG). Most heuristic search algorithms exclude all invalid solutions from the search space so that all intermediate and final solutions satisfy the constraints. However, some values of intermediate solutions that are sometimes invalid may help in finding the optimal solution. Therefore, such solutions may need to be "tolerated" in the generation process. This paper proposes a tolerance-based memetic algorithm named QSMA that employs quantum particle swarm optimization (QPSO) as a global searching operator and improved simulated annealing as a local searching operator to balance exploration and exploitation synergistically. Meanwhile, QSMA incorporates a stochastic ranking strategy to address the challenge of setting the penalty coefficient. Besides, a specific penalty function is proposed to deal with constraints, and design guidelines for penalty functions are suggested. In the experiment, the impacts of parameter settings on the performance of QSMA are investigated. Extensive experimental results show that the QSMA algorithm outperforms other methods and tools for both CAG and CCAG problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Algorithm 3
Algorithm 4
Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

Data are available on request to the authors.

Notes

  1. https://matris.sba-research.org/tools/cagen/#/workspaces.

  2. https://github.com/superjessie/CIT-WCA.

  3. https://github.com/jkunlin/FastCATool.

  4. https://github.com/jkunlin/TCA.

  5. http://www0.cs.ucl.ac.uk/staff/Yue.Jia/projects/cit_hyperheuristic/downloads/Comb_Linux_64.tar.gz.

  6. https://cse.unl.edu/~citportal/.

References

  1. Sheng Y (2013) Development of testing platform of embedded software based on combinatorial testing. Ph.D. thesis, Harbin Institute of Technology

  2. Kuhn DR, Reilly MJ (2003) An investigation of the applicability of design of experiments to software testing. In: 27th Annual NASA Goddard/IEEE software engineering workshop, 2002. Proceedings

  3. Nie C, Leung H (2011) A survey of combinatorial testing. ACM Comput Surv 43(2):1–29

    Article  MATH  Google Scholar 

  4. Wu H, Nie C, Petke J, Jia Y, Harman M (2019) A survey of constrained combinatorial testing. arXiv preprint arXiv:1908.02480

  5. Ahmed BS, Zamli KZ, Afzal W, Bures M (2017) Constrained interaction testing: a systematic literature study. IEEE Access 5:25706–25730

    Article  Google Scholar 

  6. Wu H, Nie C, Petke J, Jia Y, Harman M (2019) Comparative analysis of constraint handling techniques for constrained combinatorial testing. IEEE Trans Softw Eng 47(11):2549–2562

    Article  Google Scholar 

  7. Guo X, Song X, Zhou J (2022) A synergic quantum particle swarm optimisation for constrained combinatorial test generation. IET Softw 16(3):279–300

    Article  Google Scholar 

  8. Adam SP, Alexandropoulos S-AN, Pardalos PM, Vrahatis MN (2019) No free lunch theorem: a review. Approx Optim 57–82

  9. Kimbrough SO, Koehler GJ, Lu M, Wood DH (2008) On a feasible-infeasible two-population (fi-2pop) genetic algorithm for constrained optimization: distance tracing and no free lunch. Eur J Oper Res 190(2):310–327

    Article  MathSciNet  MATH  Google Scholar 

  10. Galinier P, Kpodjedo S, Antoniol G (2017) A penalty-based tabu search for constrained covering arrays. In: Proceedings of the genetic and evolutionary computation conference, pp 1288–1294

  11. Wang Y, Wu H, Niu X, Nie C, Xu J (2022) An adaptive penalty based parallel tabu search for constrained covering array generation. Inf Softw Technol 143:106768

    Article  Google Scholar 

  12. Runarsson TP, Yao X (2000) Stochastic ranking for constrained evolutionary optimization. IEEE Trans Evol Comput 4(3):284–294

    Article  Google Scholar 

  13. Avila-George H, Torres-Jimenez J, Gonzalez-Hernandez L, Hernández V (2013) Metaheuristic approach for constructing functional test-suites. IET Softw 7(2):104–117

    Article  Google Scholar 

  14. Jia Y, Cohen MB, Harman M, Petke J (2015) Learning combinatorial interaction test generation strategies using hyperheuristic search. In: 2015 IEEE/ACM 37th IEEE international conference on software engineering, vol. 1, pp 540–550. IEEE

  15. Garvin BJ, Cohen MB, Dwyer MB (2009) An improved meta-heuristic search for constrained interaction testing. In: 2009 1st International symposium on search based software engineering, pp 13–22. IEEE

  16. Esfandyari S, Rafe V (2018) A tuned version of genetic algorithm for efficient test suite generation in interactive t-way testing strategy. Inf Softw Technol 94:165–185

    Article  Google Scholar 

  17. Alazzawi AK, Rais HM, Basri S, Alsariera YA, Capretz LF, Balogun AO, Imam AA (2021) Habcsm: a hamming based t-way strategy based on hybrid artificial bee colony for variable strength test sets generation. arXiv preprint arXiv:2110.03728

  18. Lin J, Luo C, Cai S, Su K, Hao D, Zhang L (2015) TCA: an efficient two-mode meta-heuristic algorithm for combinatorial test generation (t). In: 2015 30th IEEE/ACM international conference on automated software engineering (ASE), pp 494–505. IEEE

  19. Lin J, Cai S, Luo C, Lin Q, Zhang H (2019) Towards more efficient meta-heuristic algorithms for combinatorial test generation. In: Proceedings of the 2019 27th ACM joint meeting on European Software engineering conference and symposium on the foundations of software engineering, pp 212–222

  20. Luo C, Lin J, Cai S, Chen X, He B, Qiao B, Zhao P, Lin Q, Zhang H, Wu W et al (2021) Autoccag: an automated approach to constrained covering array generation. In: 2021 IEEE/ACM 43rd international conference on software engineering (ICSE), pp 201–212. IEEE

  21. Fu Y, Lei Z, Cai S, Lin J, Wang H (2020) WCA: a weighting local search for constrained combinatorial test optimization. Inf Softw Technol 122:106288

    Article  Google Scholar 

  22. Wu H, Nie C, Kuo F-C, Leung H, Colbourn CJ (2014) A discrete particle swarm optimization for covering array generation. IEEE Trans Evol Comput 19(4):575–591

    Article  Google Scholar 

  23. Sheng Y, Wei C, Jiang S (2017) Constraint test cases generation based on particle swarm optimization. Int J Reliab Qual Saf Eng 24(05):1750021

    Article  Google Scholar 

  24. Wagner M, Kampel L, Simos DE (2019) Quantum-inspired evolutionary algorithms for covering arrays of arbitrary strength. In: International symposium on experimental algorithms, pp 300–316. Springer

  25. Cai X, Zhao H, Shang S, Zhou Y, Deng W, Chen H, Deng W (2021) An improved quantum-inspired cooperative co-evolution algorithm with muli-strategy and its application. Expert Syst Appl 171:114629

    Article  Google Scholar 

  26. Coello CAC (2002) Theoretical and numerical constraint-handling techniques used with evolutionary algorithms: a survey of the state of the art. Comput Methods Appl Mech Eng 191(11–12):1245–1287

    Article  MathSciNet  MATH  Google Scholar 

  27. Ahmed BS, Gambardella LM, Afzal W, Zamli KZ (2017) Handling constraints in combinatorial interaction testing in the presence of multi objective particle swarm and multithreading. Inf Softw Technol 86:20–36

    Article  Google Scholar 

  28. Li Y, Sun Z-A, Fang J-Y (2016) Generating an automated test suite by variable strength combinatorial testing for web services. J Comput Inf Technol 24(3):271–282

    Article  Google Scholar 

  29. Huang S, Cohen MB, Memon AM (2010) Repairing gui test suites using a genetic algorithm. In: 2010 Third international conference on software testing, verification and validation, pp 245–254. IEEE

  30. Guo X, Song X, Zhou J-t (2018) Effective discrete memetic algorithms for covering array generation. In: 2018 IEEE 42nd annual computer software and applications conference (COMPSAC), vol. 1, pp 298–303. IEEE

  31. Sun J, Fang W, Wu X, Palade V, Xu W (2012) Quantum-behaved particle swarm optimization: analysis of individual particle behavior and parameter selection. Evol Comput 20(3):349–393

  32. Sun J, Xu W, Liu J (2005) Parameter selection of quantum-behaved particle swarm optimization. In: International conference on natural computation, pp 543–552. Springer

  33. Chen X, Gu Q, Wang Z-Y, Chen D-X (2011) Framework of particle swarm optimization based pairwise testing. J Softw 22(12):2879–2893

    Article  Google Scholar 

  34. Garvin BJ, Cohen MB, Dwyer MB (2011) Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empir Softw Eng 16(1):61-102

    Article  Google Scholar 

  35. Alazzawi AK, Rais HM, Basri S (2019) Abcvs: An artificial bee colony for generating variable t-way test sets. Int J Adv Comput Sci Appl 10(4)

  36. Ahmed BS, Zamli KZ, Lim CP (2012) Application of particle swarm optimization to uniform and variable strength covering array construction. Appl Soft Comput 12(4):1330–1347

    Article  Google Scholar 

  37. Alsewari ARA, Zamli KZ (2012) Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support. Inf Softw Technol 54(6):553–568

    Article  Google Scholar 

  38. Wagner M, Kleine K, Simos DE, Kuhn R, Kacker R (2020) Cagen: a fast combinatorial test generation tool with support for constraints and higher-index arrays. In: 2020 IEEE international conference on software testing, verification and validation workshops (ICSTW), pp 191–200. IEEE

Download references

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China under Grant No. 62162046, the Inner Mongolia Science and Technology Project under Grant No. 2021GG0155, the Natural Science Foundation of Major Research Plan of Inner Mongolia under Grant No. 2019ZD15, and the Inner Mongolia Natural Science Foundation under Grant No. 2019GG372.

Author information

Authors and Affiliations

  1. The College of Computer Science, Inner Mongolia University, Hohhot, 010021, Inner Mongolia, China

    Xu Guo, Jian-tao Zhou & Feiyu Wang

  2. The Department of Electrical and Computer Engineering, Portland State University, Portland, OR, 97207, USA

    Xiaoyu Song

  3. The National & Local Joint Engineering Research Center of Intelligent Information Processing Technology for Mongolian, Engineering Research Center of Ecological Big Data, Ministry of Education, Inner Mongolia Engineering Laboratory for Cloud Computing and Service Software, Inner Mongolia Key Laboratory of Social Computing and Data Processing, Inner Mongolia Engineering Laboratory for Big Data Analysis Technology, Hohhot, 010021, Inner Mongolia, China

    Xu Guo & Jian-tao Zhou

Authors
  1. Xu Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoyu Song
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jian-tao Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Feiyu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian-tao Zhou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, X., Song, X., Zhou, Jt. et al. A tolerance-based memetic algorithm for constrained covering array generation. Memetic Comp. 15, 319–340 (2023). https://doi.org/10.1007/s12293-023-00392-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12293-023-00392-1

Keywords

Search

Navigation