Skip to main content

Advertisement

SpringerLink
Log in

Harmony search-based test data generation for branch coverage in software structural testing

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Test data generation is always a key task in the field of software testing. In recent years, meta-heuristic search techniques have been considered as an effective way to assist test data generation in software structural testing. In this way, some representative test cases with high-coverage capability can be picked out from program input space. Harmony search (HS) is a recently developed algorithm and has been vigorously applied to various optimization problems. In the paper, we attempt to apply harmony search algorithm to generate test data satisfying branch coverage. At the preprocessing stage, the probes used for gathering coverage information are inserted into all branches via program static analysis. At the same time, the encoding and decoding styles between a test case and a harmony are also determined in advance. At the stage of test data searching, the subset of test data that has much stronger covering ability is stored in harmony memory. During the evolution process, one part of test suite is selected and adjusted from the harmony memory, and the other part is randomly generated from input space. Once a test suite is yielded after one-round search, its coverage can be measured by fitness function in our search algorithm. In our work, a new fitness function for branch coverage is constructed by comprehensively considering branch distance and branch weight. Here, the branch weight is determined by branch information in program, that is, the nesting level of a specific branch and the predicate types in it. Subsequently, the computed coverage metric is used for updating the test suite in the next round of searching. In order to validate the effectiveness of our proposed method, eight well-known programs are used for experimental evaluation. Experimental results show that the coverage of HS-based method is usually higher than those of other search algorithms, such as simulated annealing (SA) and genetic algorithm (GA). Meanwhile, HS demonstrates greater stability than SA and GA when varying the population size or performing repeated trials. That is to say, music-inspired HS algorithm is more suitable to generate test data for branch coverage in software structural testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://z3.codeplex.com/.

  2. http://minion.sourceforge.net/.

References

  1. Littlewood B, Strigini L (2000) Software reliability and dependability: a roadmap. In: Proceedings of the 22nd international conference on software engineering, future of software engineering track, pp 175–188, June

  2. Bertolino A (2007) Software testing research: achievements, challenges, dreams. In: Proceedings of the ICSE workshop on the future of software engineering (FOSE’07), pp 85–103, May

  3. Korel B (1992) Dynamic method for software test data generation. Softw Test Verif Reliab 2(4):203-213

    Article  Google Scholar 

  4. Gupta N, Mathur AP, Soffa ML (1998) Automated test data generation using an iterative relaxation method. In: Proceedings of the SIGSOFT FSE’98, pp 231–244

  5. Offutt A, Jin Z, Pan J (1999) The dynamic domain reduction procedure for test data generation. Softw Pract Exp 29(2):167–193

    Article  Google Scholar 

  6. King JC (1976) Symbolic execution and program testing. Commun ACM 19(7):385–394

    Article  MATH  Google Scholar 

  7. Cadar C, Godefroid P, Khurshid S, Pǎsǎreanu CS, Sen K, Tillmann N, Visser W (2011) Symbolic execution for software testing in practice—preliminary assessment. In: Proceedings of the 33rd international conference on software engineering (ICSE’11), pp 1066–1071, May

  8. Gotlieb A, Botella B, Rueher M (1998) Automatic test data generation using constraint solving techniques. In: Proceedings of the ISSTA’98, pp 53–62

  9. Edvardsson J, Kamkar M (2001) Analysis of the constraint solver in una based test data generation. In: Proceedings of the ESEC/FSE’01, pp 237–245

  10. Zhang J, Xu C, Wang X (2004) Path-oriented test data generation using symbolic execution and constraint solving techniques. In: Proceedings of the international conference on software engineering and formal methods, pp 242–250

  11. Ge X, Taneja K, Xie T, Tillmann N (2011) Dyta: dynamic symbolic execution guided with static verification results. In: Proceedings of the ICSE’11, pp 992–994

  12. McMinn P (2004) Search-based software test data generation: a survey. Soft Test Verif Reliab 14:105–156

    Article  Google Scholar 

  13. McMinn P (2011) Search-based software testing: past, present and future. In: Proceedings of ICSE workshop on the search-based software testing (SBST’11), pp 153–163

  14. Tracey N, Clark J, Mander K, McDermid J (1998) An automated framework for structural test-data generation. In: Proceedings of the 13th international conference on automated software engineering (ASE’98), pp 285–288

  15. Ayari K, Bouktif S, Antoniol G (2007) Automatic mutation test input data generation via ant colony. In: Proceedings of the 9th annual conference on genetic and evolutionary computation (GECCO’07), pp 1074–1081

  16. Sthamer HH (1995) The automatic generation of software test data using genetic algorithms, PhD thesis. University of Glamorgan, November

  17. Jones BF, Sthamer HH, Eyres DE (1996) Automated structural testing using genetic algorithms. Softw Eng J 11(5):299–306

    Article  Google Scholar 

  18. Pargas RP, Harrold MJ, Peck R (1999) Automated structural testing using genetic algorithms. Softw Test Verif Reliab 9(4):263–282

    Article  Google Scholar 

  19. Harman M, McMinn P (2010) A theoretical and empirical study of search-based testing: local, global, and hybrid search. IEEE Trans Softw Eng 36(2):226–247

    Article  Google Scholar 

  20. Geem ZM, Kim J, Loganathan G (2001) A new heuristic optimization algorithm: harmony search. Simulation 76(2):60–68

    Article  Google Scholar 

  21. Geem ZM (2009) Music-inspired harmony search algorithm: theory and applications. Springer, Berlin

    Book  Google Scholar 

  22. Ammann P, Offutt J (2008) Introduction to software testing. Cambridge University Press, London

    Book  MATH  Google Scholar 

  23. Zhu H, Hall PAV, May JHR (1997) Software unit test coverage and adequacy. ACM Comput Surv 29(4):366-427

    Article  Google Scholar 

  24. Mao C (2013) Structural test data generation based on harmony search. In: Proceedings of the 4th international conference on international conference on swarm intelligence (ICSI’13), pp 353–360, June

  25. Holland JH (1992) Genetic algorithms. Sci Am 267(1):66–72

    Article  Google Scholar 

  26. Michael CC, McGraw GE, Schatz MA, Walton CC (1997) Genetic algorithms for dynamic test data generations. Technical Report RSTR-003-97-11, May

  27. Korel B (1990) Automated software test data generation. IEEE Trans Softw Eng 16(8):870–879

    Article  Google Scholar 

  28. Harman M, Hu L, Hierons R, Wegener J, Sthamer H, Baresel A, Roper M (2004) Testability transformation. IEEE Trans Softw Eng 30(1):3–16

    Article  Google Scholar 

  29. Gong D, Yao X (2012) Testability transformation based on equivalence of target statements. Neural Comput Appl 21(8):1871–1882

    Article  Google Scholar 

  30. Kirkpatrick S, Gelatt JCD, Vecchi MP (1983) Optimization by simulated annealing. Science 220(4598):671–680

    Article  MATH  MathSciNet  Google Scholar 

  31. Cohen MB, Colbourn CJ, Ling ACH (2003) Augmenting simulated annealing to build interaction test suites. In: Proceedings of the 14th international symposium on software reliability engineering (ISSRE’03), pp 394–405

  32. Shi Y, Eberhart RC (1998) A modified particle swarm optimizer. In: Proceedings of the 1998 IEEE international conference on evolutionary computation (ICEC’98), pp 69–73

  33. Windisch A, Wappler S, Wegener J (2007) Applying particle swarm optimization to software testing. In: Proceedings of the 9th annual conference on genetic and evolutionary computation (GECCO’07), pp 1121–1128

  34. Li H, Lam CP (2005) An ant colony optimization approach to test sequence generation for state-based software testing. In: Proceedings of the 5th international conference on quality software (QSIC’05), pp 255–264

  35. Sagarna R, Arcuri A, Yao X (2007) Estimation of distribution algorithms for testing object oriented software. In: Proceedings of the IEEE congress on evolutionary computation (CEC’07), pp 438–444, September

  36. Bouchachia A (2007) An immune genetic algorithm for software test data generation. In: Proceedings of the 7th international conference on hybrid intelligent systems (HIS’07), pp 84–89

  37. Zou D, Gao L, Li S, Wu J, Wang X (2010) A novel global harmony search algorithm for task assignment problem. J Syst Softw 83:1678–1688

    Article  Google Scholar 

  38. Mahdavi M, Abolhassani H (2009) Harmony k-means algorithm for document clustering. Data Min Knowl Discov 18:370–391

    Article  MathSciNet  Google Scholar 

  39. Forsati R, Mahdavi M (2010) Web text mining using harmony search. In: Geem ZW (ed) Recent advances in harmony search algorithm. Springer, Berlin

    Google Scholar 

  40. Karimi Z, Abolhassani H, Beigy H (2012) A new method of mining data streams using harmony search. J Intell Inf Syst 39(2):491–511

    Article  Google Scholar 

  41. Jafarpour N, Khayyambashi M-R (2012) Qos-aware selection of web service compositions using harmony search algorithm. J Digit Inf Manage 8(3):160–166

    Google Scholar 

  42. Rahman A, Alsewari A, Zamli KZ (2012) Design and implementation of a harmony-search-based variable-strength t-way testing strategy with constraints support. Inf Softw Technol 54(6):553–568

    Article  Google Scholar 

  43. Yang X-S (2009) Harmony search as a metaheuristic algorithm. In: Geem ZW (eds) Music-inspired harmony search algorithm: theory and applications. Springer, Berlin

    Google Scholar 

  44. Bertolino A, Mirandola R, Peciola E (1997) A case study in branch testing automation. J Syst Softw 38(1):47–59

    Article  Google Scholar 

  45. Wegener J, Baresel A, Sthamer H (2001) Evolutionary test environment for automatic structural testing. Inf Softw Technol 43(14):841–854

    Article  Google Scholar 

  46. Liu X, Liu H, Wang B, Chen P, Cai X (2005) A unified fitness function calculation rule for flag conditions to improve evolutionary testing. In: Proceedings of the 20th IEEE/ACM international conference on automated software engineering (ASE’05), pp 337–341, November

  47. Wang Y (2002) On cognitive informatics. In: Proceedings of the 1st IEEE international conference on cognitive informatics, pp 34–42, August

  48. Alba E, Chicano F (2008) Observations in using parallel and sequential evolutionary algorithms for automatic software testing. Comput Oper Res 35:3161–3183

    Article  MATH  Google Scholar 

  49. Ferrer J, Chicano F, Alba E (2012) Evolutionary algorithms for the multi-objective test data generation problem. Softw Pract Exp 42(11):1331–1362

    Google Scholar 

  50. Liang YD (2011) Introduction to Java Programming eighth edition. Pearson Education Inc., Upper Saddle River, NJ

    Google Scholar 

Download references

Acknowledgments

The author is grateful to the anonymous reviewers for providing detailed, thoughtful and constructive advice for improving the quality of paper and thanks Xinxin Yu for the help in experimental analysis. The author would also like to thank Prof. Francisco Chicano and Dr. Javier Ferrer at University of Málaga in Spain for providing some benchmark programs under test for experiments. This work was supported in part by the National Natural Science Foundation of China (NSFC) under Grant No. 60803046 and 61063013, the Natural Science Foundation of Jiangxi Province under Grant No. 2010GZS0044, the Science Foundation of Jiangxi Educational Committee under Grant No. GJJ10433, the Open Foundation of State Key Laboratory of Software Engineering under Grant No. SKLSE2010-08-23, and the Program for Outstanding Young Academic Talent in Jiangxi University of Finance and Economics.

Author information

Authors and Affiliations

  1. School of Software and Communication Engineering, Jiangxi University of Finance and Economics, Nanchang, 330013, China

    Chengying Mao

Authors
  1. Chengying Mao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Chengying Mao.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Mao, C. Harmony search-based test data generation for branch coverage in software structural testing. Neural Comput & Applic 25, 199–216 (2014). https://doi.org/10.1007/s00521-013-1474-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-013-1474-z

Keywords

Search

Navigation