Skip to main content
Springer Nature Link
Log in

Combinatorial test list generation based on Harmony Search Algorithm

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Combinatorial test case generation faces a problem on how to reduce the test cases by uncover the unnecessary test cases. So, there is a need for expert applications or strategies that generate the most optimum test cases keeping in mind the most important combinations. Complementing existing work on combinatorial test case generation strategies, also known as t-way testing strategies (i.e., where t represents interaction degree), this paper presents the design and the implementation of a new combinatorial test list generation strategy based on Harmony Search (HS) algorithm, called General T-way Harmony Search-based Strategy (GTHS). HS has been chosen to be the main engine of test generation because it could balance between intensification and diversification. Benchmarking experimental results show that GTHS produces competitive results as compared to other existing well-known optimization-based strategies and provides the support for high combination degrees (i.e., t ≤ 12).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

(Geem and Kim 2001; Geem and Park 2006)

Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Afzal W, Torkar R, Feldt R (2009) A systematic review of search-based testing for non-functional system properties. Inf Softw Technol 51(6):957–976. https://doi.org/10.1016/j.infsof.2008.12.005

    Article  Google Scholar 

  • Ahmed BS, Kamal Z, Zamli, Chee Peng Lim (2011) Constructing a T-way interaction test suite using the particle swarm optimization approach. Int J Innov Comput Inf Control 7(11):1–10

    Google Scholar 

  • Almohammedi AA, Noordin NK, Sali A, Hashim F, Balfaqih M (2017) An adaptive multi-channel assignment and coordination scheme for IEEE 802.11 P/1609.4 in vehicular Ad-Hoc networks. IEEE Access 6:2781–2802

    Article  Google Scholar 

  • Almohammedi AA, Noordin NK, Sali A, Hashim F, Jabbar WA, Saeed S (2019) Modeling and analysis of IEEE 1609.4 MAC in the presence of error-prone channels. Int J Electr Comput Eng 9(5):3531–3541

    Google Scholar 

  • Alsewari ARA, Zamli KZ (2012) A harmony search based pairwise sampling strategy for combinatorial testing. Int J Phys Sci 7(7):1062–1072

    Google Scholar 

  • Alsewari AA, Har HC, Homaid AAB, Nasser AB, Zamli KZ, Tairan NM (2017) Test cases minimization strategy based on flower pollination algorithm. In: International conference of reliable information and communication technology. Springer, Cham, pp 505–512

    Google Scholar 

  • Angeline P (1998) Evolutionary optimization versus particle swarm optimization: philosophy and performance differences. In: Porto V, Saravanan N, Waagen D, Eiben A (eds) Evolutionary programming VII, vol 1447. Springer, Berlin, pp 601–610

    Chapter  Google Scholar 

  • Arshem J (2010) TVG. http://sourceforge.net/projects/tvg

  • Balfaqih M, Nordin R, Balfaqih Z, Haseeb S, Hashim A (2015) An evaluation of IEEE 802.11 mac layer handoff process in capwap centralized WLAN. J Theor Appl Inf Technol 71(3):468–479

    Google Scholar 

  • Balfaqih M, Ismail M, Nordin R, Balfaqih Z (2017) Handover performance evaluation of centralized and distributed network-based mobility management in vehicular urban environment. In: Paper presented at the 2017 9th IEEE-GCC conference and exhibition (GCCCE)

  • Bansal JC, Jadon SS, Tiwari R, Kiran D, Panigrahi BK (2017) Optimal power flow using artificial bee colony algorithm with global and local neighborhoods. Int J Syst Assur Eng Manag 8(4):2158–2169. https://doi.org/10.1007/s13198-014-0321-7

    Article  Google Scholar 

  • Brownlie R, Prowse J, Phadke MS (1992) Robust testing of AT&T PMX/StarMAIL using oats. AT&T Techn J 71(3):41–47

    Article  Google Scholar 

  • Bryce RC, Colbourn CJ, Cohen MB (2005) A framework of greedy methods for constructing interaction test suites. In: Proceedings of the 27th international conference on software engineering. ACM, pp 146–155

  • Chen X, Gu Q, Li A, Chen D (2009) Variable strength interaction testing with an ant colony system approach. In: Paper presented at the proceedings of the 2009 16th Asia-Pacific software engineering conference

  • Cheng CS (1980) Orthogonal arrays with variable numbers of symbols. Ann Stat 8(2):447–453. http://www.jstor.org/stable/2240546

    MathSciNet  MATH  Google Scholar 

  • Cohen MB (2004) Designing test suites for software interaction testing. (Ph.D). University of Auckland, New Zealand

    Google Scholar 

  • Cohen DM, Dalal SR, Fredman ML, Patton GC (1997) The AETG system: an approach to testing based on combinatorial design. IEEE Trans Softw Eng 23(7):437–444

    Article  Google Scholar 

  • Cohen MB, Gibbons PB, Mugridge WB, Colbourn CJ (2003) Constructing test suites for interaction testing. In: Paper presented at the proceedings of the 25th international conference on software engineering, Portland, Oregon USA

  • Cohen MB, Colbourn CJ, Ling ACH (2008) Constructing strength three covering arrays with augmented annealing. Discret Math 308(13):2709–2722

    Article  MathSciNet  Google Scholar 

  • Dalal SR, Jain A, Karunanithi N, Leaton J, Lott CM, Patton GC, Horowitz BM (1999) Model-based testing in practice. In: Paper presented proceedings of the 1999 international conference on at the software engineering

  • Danziger P, Mendelsohn E, Moura L, Stevens B (2009) Covering arrays avoiding forbidden edges. Theor Comput Sci 410(52):5403–5414. https://doi.org/10.1016/j.tcs.2009.07.057

    Article  MathSciNet  MATH  Google Scholar 

  • Dolores RW, Kuhn DR (2001) Failure modes in medical device software: an analysis of 15 years of recall data. Int J Reliab Qual Saf Eng 8(04):351–371

    Article  Google Scholar 

  • Dongardive J, Patil A, Bir A, Jamkhedkar S, Abraham S (2010) Finding motifs using harmony search. In: Paper presented at the proceedings of the international symposium on biocomputing

  • Emadi SP, Maleki H, Honari M (2011) Harmony search for finding the best hamiltony tour in Iran. GSTF J Comput 1(2):239–244

    Article  Google Scholar 

  • Forbes M, Lawrence J, Lei Y, Kacker RN, Kuhn DR (2008) Refining the in-parameter-order strategy for constructing covering arrays. NIST J Res 113(5):287–297

    Article  Google Scholar 

  • Forsati R, Haghighat AT, Mahdavi M (2008) Harmony search based algorithms for bandwidth-delay-constrained least-cost multicast routing. Comput Commun 31(10):2505–2519

    Article  Google Scholar 

  • Geem ZW (2008) Novel derivative of harmony search algorithm for discrete design variables. Appl Math Comput 199(1):223–230

    MathSciNet  MATH  Google Scholar 

  • Geem ZW (2009) Hydraulic analysis of water distribution network using harmony search. In: Paper presented at the world environmental and water resources congress 2009: Great Rivers

  • Geem Z, Kim J (2001) A new Heuristic optimization algorithm: harmony search. Simulation 76(2):60

    Article  Google Scholar 

  • Geem Z, Park Y (2006) Harmony search for layout of rectilinear branched networks. WSEAS Trans Syst 5(6):1349–1354

    Google Scholar 

  • Geem Z, Lee K, Park Y (2005) Application of harmony search to vehicle routing. Am J Appl Sci 2(12):1552–1557

    Article  Google Scholar 

  • Ghazi SA, Ahmed MA (2003) Pair-wise test coverage using genetic algorithms. In: Paper presented at the evolutionary computation, 2003. CEC '03. The 2003 Congress

  • Huang M, Dong H-y, Wang X-w, Zheng B-l, Ip W (2009) Guided variable neighborhood harmony search for integrated charge planning in primary steelmaking processes. In: Paper presented at the Proceedings of the first ACM/SIGEVO summit on genetic and evolutionary computation

  • Itkonen J, Lassenius C (2012) The role of the tester's knowledge in exploratory software testing. IEEE Trans Softw Eng 99:1–20

    Google Scholar 

  • Jabbar WA, Ismail M, Nordin R, Ramli RM (2017, 28–30 Nov. 2017). EMA-MPR: energy and mobility-aware multi-point relay selection mechanism for multipath OLSRv2. In: Paper presented at the 2017 IEEE 13th Malaysia international conference on communications (MICC)

  • Karahan H, Gurarslan G, Geem ZW (2012) Parameter estimation of the nonlinear Muskingum flood-routing model using a hybrid harmony search algorithm. J Hydrol Eng 18(3):352–360

    Article  Google Scholar 

  • Kaveh A, Talatahari S (2009) Particle swarm optimizer, ant colony strategy and harmony search scheme hybridized for optimization of truss structures. Comput Struct 87(5–6):267–283

    Article  Google Scholar 

  • Kim JH, Geem ZW, Kim ES (2001) Parameter estimation of the nonlinear Muskingum model using harmony search. J Am Water Resour Assoc 37(5):1131–1138. https://doi.org/10.1111/j.1752-1688.2001.tb03627.x

    Article  Google Scholar 

  • Kuhn DR, Wallace DR, Gallo AM Jr (2004) Software fault interactions and implications for software testing. IEEE Trans Softw Eng 30(6):418–421. https://doi.org/10.1109/TSE.2004.24

    Article  Google Scholar 

  • Lazić L, Velašević D (2004) Applying simulation and design of experiments to the embedded software testing process. Softw Test Verif Reliab 14(4):257–282

    Article  Google Scholar 

  • Lee KS, Geem ZW (2004) A new structural optimization method based on the harmony search algorithm. Comput Struct 82(9–10):781–798

    Article  Google Scholar 

  • Lei Y, Tai KC (1998) In-parameter-order: a test generation strategy for pairwise testing. In: Paper presented at the proceedings of the 3rd IEEE Intl. High-Assurance Systems Engineering Symp, Washington, DC, USA

  • Lei Y, Kacker R, Kuhn D, Okun V, Lawrence J (2008) IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing. Softw Test Verif Reliabil 18(3):125–148. https://doi.org/10.1002/stvr.v18:3

    Article  Google Scholar 

  • Mandl R (1985) Orthogonal Latin squares: an application of experiment design to compiler testing. In: Paper presented at the communications of the ACM, New York, NY, USA

  • Mao C, Xiao L, Yu X, Chen J (2015) Adapting ant colony optimization to generate test data for software structural testing. Swarm Evolut Comput 20(0):23–36. https://doi.org/10.1016/j.swevo.2014.10.003

    Article  Google Scholar 

  • Memon AM, Soffa ML (2003) Regression testing of GUIs. In: Paper presented at the ACM SIGSOFT software engineering notes

  • Mohsen AM, Khader AT, Ghallab A (2009) The effect of using different thermodynamic models with harmony search algorithm in the accuracy of RNA secondary structure prediction. In: Paper presented at the soft computing and pattern recognition, 2009. SOCPAR'09. International Conference

  • Nurmela KJ, Östergård PR (1993) Constructing covering designs by simulated annealing. Princeton, Citeseer

    MATH  Google Scholar 

  • Pargas, R. P., Harrold, M. J., & Peck, R. R. (1999). Test-data generation using genetic algorithms. Software Testing, Verification and Reliability. 9(4), 263–282. https://doi.org/10.1002/(SICI)1099-1689(199912)9:4%3C263::AID-STVR190%3E3.0.CO;2-Y

    Article  Google Scholar 

  • Parouha RP, Das KN (2016) A novel hybrid optimizer for solving economic load dispatch problem. Int J Electr Power Energy Syst 78:108–126. https://doi.org/10.1016/j.ijepes.2015.11.058

    Article  Google Scholar 

  • Sabar NR, Ayob M, Kendall G, Qu R (2015) The automatic design of hyper-heuristic framework with gene expression programming for combinatorial optimization problems. IEEE Trans Evolut Comput 309(325):3

    Google Scholar 

  • Salih SQ, Alsewari AA (2019) A new algorithm for normal and large-scale optimization problems: nomadic people optimizer. Neural Comput Appl. https://doi.org/10.1007/s00521-019-04575-1

    Article  Google Scholar 

  • Shiba T, Tsuchiya T, Kikuno T (2004) Using artificial life techniques to generate test cases for combinatorial testing. In: Paper presented at the proceedings of the 28th annual international computer software and applications conference. https://doi.org/10.1109/CMPSAC.2004.1342808

  • Stardom J (2001) Metaheuristics and the search for covering and packing array. Master of Scienc Master thesis. Simon Fraser University, Canada

  • Wang L, Pan Q-K, Tasgetiren MF (2010) Minimizing the total flow time in a flow shop with blocking by using hybrid harmony search algorithms. Expert Syst Appl 37(12):7929–7936

    Article  Google Scholar 

  • Williams AW (2000) Determination of test configurations for pair-wise interaction coverage. In: Paper presented at the proceedings of the IFIP TC6/WG6.1 13th international conference on testing communicating systems: tools and techniques

  • Yadav A, Deep K (2014) An efficient co-swarm particle swarm optimization for non-linear constrained optimization. J Comput Sci 5(2):258–268. https://doi.org/10.1016/j.jocs.2013.05.011

    Article  Google Scholar 

  • Yi J, Adnane A, David S, Parrein B (2011) Multipath optimized link state routing for mobile ad hoc networks. Ad Hoc Netw 9(1):28–47

    Article  Google Scholar 

  • Yilmaz C, Cohen MB, Porter A (2004) Covering arrays for efficient fault characterization in complex configuration spaces. ACM SIGSOFT Softw Eng Notes 29(4):45–54. https://doi.org/10.1145/1013886.1007519

    Article  Google Scholar 

  • Younis MI, Zamli KZ (2009a) Assessing combinatorial interaction strategy for reverse engineering of combinational circuits. In: Paper presented at the industrial electronics and applications, 2009. ISIEA 2009. IEEE Symposium

  • Younis MI, Zamli KZ (2009b) ITTW: T-way minimization strategy based on intersection of Tuples. In: Paper presented at the industrial electronics and applications, 2009. ISIEA 2009. IEEE symposium, 4–6 Oct

  • Younis MI, Zamli KZ (2010a) MC-MIPOG: a parallel t-way test generation strategy for multicore systems. ETRI J 32(1):73–83

    Article  Google Scholar 

  • Younis MI, Zaml KZ (2010b) A strategy for automatic quality signing and verification processes for hardware and software testing. Adv Softw Eng 2010:323429. https://doi.org/10.1155/2010/323429

    Article  Google Scholar 

  • Younis MI, Zamli KZ, Isa NAM (2012) Adopting systematic quality signing and verification processes for sequential hardware testing. Invent Impact Struct Des 2012(1):1–4

    Google Scholar 

  • Zamli KZ, Klaib MFJ, Younis MI, Isa NAM, Abdullah R (2011) Design and implementation of a t-way test data generation strategy with automated execution tool support. Inf Sci 181(9):1741–1758. https://doi.org/10.1016/j.ins.2011.01.002

    Article  Google Scholar 

  • Zamli KZ, Alkazemi BY, Kendall G (2016) A Tabu Search hyper-heuristic strategy for t-way test suite generation. Appl Soft Comput 44:57–74. https://doi.org/10.1016/j.asoc.2016.03.021

    Article  Google Scholar 

  • Zarei O, Fesanghary M, Farshi B, Saffar RJ, Razfar M (2009) Optimization of multi-pass face-milling via harmony search algorithm. J Mater Process Technol 209(5):2386–2392

    Article  Google Scholar 

  • Zeng B, Dong Y (2016) An improved harmony search based energy-efficient routing algorithm for wireless sensor networks. Appl Soft Comput 41:135–147

    Article  Google Scholar 

Download references

Acknowledgements

The research supported by Universiti Malaysia Pahang (UMP) under the grant scheme number UMP (RDU190334): A Novel Hybrid Harmony Search Algorithm with Nomadic People Optimizer Algorithm for Global Optimization and Feature Selection, and (Ref: PRGS/1/2019/ICT02/UMP/02/1) IoT based Intelligent Combinatorial Test Cases Generator System Based on Kidney Inspired Algorithm with Opposition Approach.

Author information

Authors and Affiliations

  1. Faculty of Computing, College of Computing and Applied Sciences, Universiti Malaysia Pahang, 26300, Gambang, Pahang, Malaysia

    AbdulRahman A. Alsewari & Kamal Z. Zamli

  2. IBM Centre of Excellence, Universiti Malaysia Pahang, 26300, Gambang, Pahang, Malaysia

    AbdulRahman A. Alsewari

  3. Business Information and Technology, The University of Memphis, Memphis, USA

    Robin Poston

  4. Computer and Networks Engineering Department, Jeddah University, Jeddah, 23218, Saudi Arabia

    Mohammed Balfaqih

  5. Automobile Transportation Department, South Ural State University, Chelyabinsk, 454080, Russia

    Mohammed Balfaqih

  6. College of Computer Science and Engineering, Taibah University, Medina, 41477, Saudi Arabia

    Khalid S. Aloufi

Authors
  1. AbdulRahman A. Alsewari
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Robin Poston
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Kamal Z. Zamli
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Mohammed Balfaqih
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Khalid S. Aloufi
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to AbdulRahman A. Alsewari.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Alsewari, A.A., Poston, R., Zamli, K.Z. et al. Combinatorial test list generation based on Harmony Search Algorithm. J Ambient Intell Human Comput 13, 3361–3377 (2022). https://doi.org/10.1007/s12652-020-01696-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-020-01696-7

Keywords