Skip to main content

Advertisement

SpringerLink
Log in

Bio-inspired metaheuristics: evolving and prioritizing software test data

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Software testing is both a time and resource-consuming activity in software development. The most difficult parts of software testing are the generation and prioritization of test data. Principally these two parts are performed manually. Hence introducing an automation approach will significantly reduce the total cost incurred in the software development lifecycle. A number of automatic test case generation (ATCG) and prioritization approaches have been explored. In this paper, we propose two approaches: (1) a pathspecific approach for ATCG using the following metaheuristic techniques: the genetic algorithm (GA), particle swarm optimization (PSO) and artificial bee colony optimization (ABC); and (2) a test case prioritization (TCP) approach using PSO. Based on our experimental findings, we conclude that ABC outperforms the GA and PSO-based approaches for ATC.G Moreover, the results for PSO on TCP arguments demonstrate biased applicability for both small and large test suites against random, reverse and unordered prioritization schemes. Therefore, we focus on conducting a comprehensive and exhaustive study of the application of metaheuristic algorithms in solving ATCG and TCP problems in software engineering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Detailed MATLAB code can be provided by the authors upon prior request for academic use only.

  2. A test requirement identifies what needs to be tested, including conditions, business logic, and functional and non-functional benchmarks.

  3. Detailed MATLAB code for PSO for TCP can be provided by the authors upon prior request for academic use only.

References

  1. Tassey G (2002) The economic impacts of inadequate infrastructure for software testing. National Institute of Standards and Technology, RTI Project 7007

  2. Beizerm B (1990) Software Testing Techniques, Van Nostrand Reinhold. New York

  3. Rothermel G, Untch RH, Chu C, Harrold MJ (1999) Test case prioritization: An empirical study. In: Proceedings IEEE international conference on software maintenance, (ICSM’99). IEEE, pp 179–188

  4. Hou Y, Zhao C, Liao Y (2006) A new method of test generation for sequential circuits. In: International conference on communications, circuits and systems proceedings. IEEE, pp 2181–2185

  5. Liang Y, Liu L, Wang D, Wu R (2010) Optimizing particle swarm optimization to solve knapsack problem Information computing and applications. Springer, pp 437–443

  6. Liu H, Sun S, Abraham A (2006) Particle swarm approach to scheduling work-flow applications in distributed data-intensive computing environments. In: Sixth international conference on intelligent systems design and applications, ISDA’06, pp 661– 666

    Chapter  Google Scholar 

  7. Lope HS, Coelho LS (2005) Particle swarn optimization with fast local search for the blind traveling salesman problem. In: Fifth international conference on hybrid intelligent systems, HIS’05. IEEE, pp 245–250

  8. Zhao F, Zhang Q, Yang Y (2006) An improved particle swarm optimization-based approach for production scheduling problems. In: IEEE international conference on mechatronics and automation. IEEE, 2279–2283

    Google Scholar 

  9. Do H, Rothermel G (2006) On the use of mutation faults in empirical assessments of test case prioritization techniques. IEEE Trans Softw Eng 32:733–752

    Article  Google Scholar 

  10. Elbaum S, Malishevsky AG, Rothermel G (2002) Test case prioritization: A family of empirical studies. IEEE Trans Softw Eng 28:159–182

    Article  Google Scholar 

  11. Arcuri A, Briand L (2012) Formal analysis of the probability of interaction fault detection using random testing. IEEE Trans Softw Eng 38:1088–1099

    Article  Google Scholar 

  12. Mills HD, Dyer M, Linger RC (1987) Cleanroom software engineering. IEEE Softw 4:19

    Article  Google Scholar 

  13. Voas J, Morell L, Miller K (1991) Predicting where faults can hide from testing. IEEE Softw 8:41–48

    Article  Google Scholar 

  14. Bird DL, Munoz CU (1983) Automatic generation of random self-checking test cases. IBM Syst J 22:229–245

    Article  Google Scholar 

  15. Ferguson R, Korel B (1996) The chaining approach for software test data generation. ACM Trans Softw Eng Methodol (TOSEM) 5:63–86

    Article  Google Scholar 

  16. Chang K-H, Cross IIJH, Carlisle WH, Brown DB (1992) A framework for intelligent test data generation. J Intell Robot Syst 5:147–165

    Article  Google Scholar 

  17. Korel B (1996) Automated test data generation for programs with procedures ACM SIGSOFT software engineering notes. ACM, pp 209–215

  18. Korel B (1990) A dynamic approach of test data generation. In: Proceedings, conference on software maintenance. IEEE, pp 311–317

  19. McMinn P (2004) Search-based software test data generation: a survey. Software Testing Verification and Reliability 14:105–156

    Article  Google Scholar 

  20. Clarke LA (1976) A system to generate test data and symbolically execute programs. IEEE Trans Softw Eng :215–222

  21. Howden WE (1977) Symbolic testing and the DISSECT symbolic evaluation system. IEEE Trans Softw Eng :266–278

  22. Ramamoorthy CV, Ho S-B, Chen WT (1976) On the automated generation of program test data. IEEE Trans Softw Eng :293–300

  23. Shan J-H, Wang J, Qi Z-C (2004) Survey on path-wise automatic generation of test data. Acta Electron Sin 32:109–113

    Google Scholar 

  24. Malhotra R, Garg M (2011) An adequacy based test data generation technique using genetic algorithms. J Inf Process Syst 7:363–384

    Article  Google Scholar 

  25. Michael CC, McGraw G, Schatz MA (2001) Generating software test data by evolution. IEEE Trans Softw Eng 27:1085–1110

    Article  Google Scholar 

  26. Fraser G, Arcuri A (2014) A large-scale evaluation of automated unit test generation using EvoSuite. ACM Trans Softw Eng Methodol (TOSEM) 24:8

    Article  Google Scholar 

  27. Tracey N, Clark J, Mander K (1998) Automated program flaw finding using simulated annealing ACM SIGSOFT software engineering notes. ACM, pp 73–81

  28. Malhotra R, Khari M (2013) Heuristic search-based approach for automated test data generation: a survey. International Journal of Bio-Inspired Computation 5:1–18

    Article  Google Scholar 

  29. Do H, Rothermel G, Kinneer A (2004) Empirical studies of test case prioritization in a JUnit testing environment. In: 5th International symposium on software reliability engineering ISSRE. IEEE, pp 113–124

  30. Kim J-M, Porter A (2002) A history-based test prioritization technique for regression testing in resource constrained environments Proceedings of the 24rd international conference on software engineering, ICSE. IEEE, pp 119–129

  31. Rothermel G, Untch RH, Chu C, Harrold MJ (2001) Prioritizing test cases for regression testing. IEEE Trans Softw Eng 27:929–948

    Article  Google Scholar 

  32. Leon D, Podgurski A (2003) A comparison of coverage-based and distribution-based techniques for filtering and prioritizing test cases. In: 14th international symposium on software reliability engineering, ISSRE. IEEE, pp 442–453

  33. Mirarab S, Tahvildari L (2008) An empirical study on bayesian network-based approach for test case prioritization. In: 1st International conference on software testing, verification, and validation. IEEE, pp 278–287

  34. Yoo S, Harman M (2012) Regression testing minimization, selection and prioritization: a survey. Software testing, verification and reliability 22:67–120

    Article  Google Scholar 

  35. Li Z, Harman M, Hierons RM (2007) Search algorithms for regression test case prioritization. IEEE Trans Softw Eng 33:225–237

    Article  Google Scholar 

  36. Kaur DA, Goyal S (2011) A bee colony optimization algorithm for code coverage test suite prioritization. Int J Eng Sci Technol 1:2786–2795

    Google Scholar 

  37. Catal C, Mishra D (2013) Test case prioritization: a systematic mapping study. Softw Qual J 21:445–478

    Article  Google Scholar 

  38. Jacob TP, Ravi T (2013) Optimization of test cases by prioritization. J Comput Sci 9:972

    Article  Google Scholar 

  39. Singh Y, Kaur A, Suri B (2010) Test case prioritization using ant colony optimization. ACM SIGSOFT Software Engineering Notes 35:1–7

    Google Scholar 

  40. Ahmed AA, Shaheen M, Kosba E (2012) Software testing suite prioritization using multi-criteria fitness function. In: 222nd International conference on computer theory and applications (ICCTA). IEEE, pp 160–166

  41. Chen Y, Zhong Y (2008) Automatic path-oriented test data generation using a multi-population genetic algorithm. In: Fourth international conference on natural computation, ICNC’08. IEEE, pp 566–570

  42. Mohapatra D (2011) GA Based Test Case Generation Approach for Formation of Efficient Set of Dynamic Slices. International Journal on Computer Science and Engineering (IJCSE) 3:

  43. Pargas RP, Harrold MJ, Peck RR (1999) Test-data generation using genetic algorithms. Software Testing Verification and Reliability 9:263–282

    Article  Google Scholar 

  44. Berndt D, Fisher J, Johnson L et al (2003) Breeding software test cases with genetic algorithms. In: Proceedings of the 36th annual hawaii international conference on system sciences

  45. Ahmed M A, Hermadi I (2008) GA-Based multiple paths test data generator. Comput Oper Res 35:3107–3124

    Article  Google Scholar 

  46. Mukesh MOPS (2015) Generating and prioritizing optimal paths using ant colony optimization. Computational Ecology and Software 5:1

    Google Scholar 

  47. Watkins AL (1995) The automatic generation of test data using genetic algorithms. In: Proceedings of the 4th software quality conference, pp 300–309

    Google Scholar 

  48. Flipcart Shop Flipcart, availabte: www.flipcart.com, accessed june 2016

  49. Arcuri A, Briand L (2014) A hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability 24:219– 250

    Article  Google Scholar 

  50. Rothermel G, Harrold MJ (1996) Analyzing regression test selection techniques. IEEE Trans Softw Eng 22:529–551

    Article  Google Scholar 

  51. Di Caro G, Dorigo M (1998) Antnet: Distributed stigmergetic control for communications networks. J Artif Intell Res :317– 365

  52. Dorigo M, Maniezzo V, Colorni A (1996) Ant system: optimization by a colony of cooperating agents. IEEE Transactions on Systems, Man, and Cybernetics 26:29–41

    Article  Google Scholar 

  53. Li H, Lam CP (2004) Software Test Data Generation using Ant Colony Optimization. In: International conference on computational intelligence, pp 1–4

    Google Scholar 

  54. Ayari K, Bouktif S, Antoniol G (2007) Automatic mutation test input data generation via ant colony. In: Proceedings of the 9th annual conference on Genetic and evolutionary computation. ACM, pp 1074–1081

  55. Parpinelli RS, Lopes HS, Freitas AA (2002) Data mining with an ant colony optimization algorithm. IEEE Trans Evol Comput 6:321–332

    Article  MATH  Google Scholar 

  56. Zhao P, Zhao P, Zhang X (2006) A new ant colony optimization for the knapsack problem. In: CAIDCD’06 7th international conference on computer-aided industrial design and conceptual design. IEEE, pp 1–3

  57. Atif M (2016) Software Benchmark repository for TeraPaint3. http://www.cs.umd.edu/~atif/Benchmarks/common/TerpPaint3-fault_matrix.txt. Accessed 1 June 2016

  58. Atif M (2016) Software Benchmark repository for TeraPresent3. http://www.cs.umd.edu/~atif/Benchmarks/common/TerpPresent3-fault_matrix.txt. Accessed 20 June 2016

  59. Atif M (2016) Software Benchmark repository for TerpSpreadSheet3, available. http://www.cs.umd.edu/~atif/Benchmarks/common/TerpSpreadSheet3-fault_matrix.txt

  60. Krishnamoorthi R, Mary SASA (2009) Regression test suite prioritization using genetic algorithms. International Journal of Hybrid Information Technology 2:35–52

    Google Scholar 

  61. Srivastava PR (2008) Test case prioritization. Journal of Theoretical and Applied Information Technology 4:178–181

    Google Scholar 

  62. Kaur A, Bhatt D (2011) Hybrid particle swarm optimization for regression testing. Int J Comput Sci Eng 3:1815–1824

    Google Scholar 

  63. Kaur A, Bhatt D (2011) Particle swarm optimization with cross-over operator for prioritization in regression testing. Int J Comput Appl 27:27–34

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, School of Information & Communication Technology, Gautam Buddha University, Greater Noida, Uttar- Pradesh, India

    Mukesh Mann & Pradeep Tomar

  2. Department of Computer Science and Engineering, Guru Jambheshwar University of Science and Technology, Hisar, Haryana, India

    Om Prakash Sangwan

Authors
  1. Mukesh Mann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pradeep Tomar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Om Prakash Sangwan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mukesh Mann.

Appendices

Appendix 1: Small application test suite- fault matrix

Only the main modules are provided in this paper. Detailed source code is available from the authors upon prior request.

Table 11
Full size table

Large real application test suite: data can be accessed at [57].

Appendix 2: Objective function formulation for TeraPaint3

The following code is implemented in MATLAB and forms the objective function. The objective function is given as an input to the PSO algorithm. Note that the lower and upper bounds in the PSO algorithm depend on the system under test. Because TeraPaint3 has 424 test cases, the lower and upper bounds are set to [1] and [424], respectively.

Table 12
Full size table

The APFD scores for the other two applications, TeraPresent3 and TeraSpreadSheet3, are calculated in a similar manner.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mann, M., Tomar, P. & Sangwan, O.P. Bio-inspired metaheuristics: evolving and prioritizing software test data. Appl Intell 48, 687–702 (2018). https://doi.org/10.1007/s10489-017-1003-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-017-1003-3

Keywords

Search

Navigation