Skip to main content
Springer Nature Link
Log in

Metaheuristic Optimisation and Mutation-Driven Test Data Generation

  • Chapter
  • First Online:
Computational Intelligence and Quantitative Software Engineering

Part of the book series: Studies in Computational Intelligence ((SCI,volume 617))

  • 699 Accesses

Abstract

Metaheuristic optimisation techniques can be used in combination with mutation analysis to generate test data that is effective at finding faults and reduces the human effort involved in software testing. This chapter describes and evaluates various different metaheuristic techniques and considers their underlying properties in relation to test data generation. This represents the first attempt to bring together, compare and review ideas and research related to mutation analysis and metaheuristic optimisation. The intention is that by considering these application areas together, we can appreciate and understand important aspects of their strengths and weaknesses. This will allow us to make suggestions with regards to the ways in which they may be used together for maximum effectiveness and efficiency.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Talbi, E.-G.: Metaheuristics: From Design to Implementation. Wiley, Hoboken (2009)

    Book  MATH  Google Scholar 

  2. Yang, X.-S.: Metaheuristic optimization: algorithm analysis and open problems. Lect. Notes Comp. Sci. 6630, 21–32 (2011)

    Article  Google Scholar 

  3. McMinn, P.: Search-based software test data generation: a survey. Softw. Test. Verif. Reliab. 14, 105–156 (2004)

    Article  Google Scholar 

  4. Bianchi, L., Dorigo, M., Gambardella, L.M., Gutjahr, W.J.: A Survey on metaheuristics for stochastic combinatorial optimization. Nat. Comput. 8, 239–287 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  5. Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37, 649–678 (2011)

    Article  Google Scholar 

  6. Offutt, A.J.: Investigations of the software testing coupling effect. ACM Trans. Softw. Eng. Methodol. 1, 5–20 (1992)

    Article  Google Scholar 

  7. Ghiduk, A.S.: Using evolutionary algorithms for higher-order mutation testing. Int. J. Comp. Sci. Issues 11, 93–104 (2014)

    Google Scholar 

  8. Mishra, K.K., Tiwari, S., Kumar, A., Misra, A.K.: An approach for mutation testing using elitist genetic algorithm. In: Proceedings of IEEE International Conference Computer Science Information Technology 426–429 (2010)

    Google Scholar 

  9. Baudry, B., Fleurey, F., Jézéquel, J.-M., Le Traon, Y.: From genetic to bacteriological algorithms for mutation-based testing. Softw. Test. Verif. Reliab. 15, 73–96 (2005)

    Article  Google Scholar 

  10. Bottaci, L.: A genetic algorithm fitness function for mutation testing. In: Proceedings of International Works. Software Engineering Metaheuristic Innovative Algorithms, pp. 3–7 (2001)

    Google Scholar 

  11. Fraser, G., Zeller, A.: Mutation driven generation of unit tests and oracles. In: Proceedings of 19th International Symposium Software Testing Analysis, pp. 147–158 (2010)

    Google Scholar 

  12. Papadakis, M., Malevris, N.: Searching and generating test inputs for mutation testing. SpringerPlus 2 (2013)

    Google Scholar 

  13. Korel, B.: Automated software test data generation. IEEE Trans. Softw. Eng. 16, 870–879 (1990)

    Article  Google Scholar 

  14. Kaur, A., Goyal, S.: A bee colony optimization algorithm for fault coverage based regression test suite prioritization. Int. J. Adv. Sci. Tech. 29, 17–30 (2011)

    Google Scholar 

  15. Ayari, K., Bouktif, S., Antoniol, G.: Automatic mutation test input data generation via ant colony. In: Proceedings of 9th Annual Conference Genetic Evolutionary Computation, pp. 1074–1081 (2007)

    Google Scholar 

  16. Myers, G.J., Sandler, C., Badgett, T.: The Art of Software Testing. Wiley, Hoboken (2012)

    Book  Google Scholar 

  17. Blair, M., Obenski, S., Bridickas, P.: Patriot missile software problem. Technical report GAO/IMTEC-92-26, United States General Accounting Office (2002)

    Google Scholar 

  18. Heusser, M.: Software testing lessons learned from Knight Capital Fiasco. CIO Magazine (2012). http://www.cio.com/article/713628/Software_Testing_Lessons_Learned_From_Knight_Capital_Fiasco/. Cited 28 Sep 2014

  19. Bezier, B.: Software Testing Techniques. Van Nostrand Reinhold, New York (1990)

    Google Scholar 

  20. Jay, F., Mayer, R.: IEEE standard glossary of software engineering terminology. Technical report 610.12-1990, IEEE (1990)

    Google Scholar 

  21. May, P.S.: Test data generation: two evolutionary approaches to mutation testing. Ph.D. thesis, Department of Computer Science, University of Kent, Canterbury, UK (2007)

    Google Scholar 

  22. Pacheco, C., Lahiri, S., Ball, T.: Finding errors in .NET with feedback-directed random testing. Technical report MSR-TR-2008-29, Microsoft Research (2008)

    Google Scholar 

  23. Sun Microsystems: Java Compatability Kit 6b User’s Guide. Sun Microsystems (2012). http://openjdk.java.net/groups/conformance/docs/JCK6bUsersGuide/JCK6b_Users_Guide.pdf. Cited 28 Sep 2014

  24. Oracle: Java Bug Database. Oracle (2012). http://bugs.sun.com. Cited 28 Sept 2014

  25. Dijkstra, E.W.: Notes on structured programming. In: Dahl, O.J., Dijkstra, E.W., Hoare, C.A.R. (eds.) Structured Programming, pp. 1–82. Academic Press Ltd., London (1972)

    Google Scholar 

  26. Mahmood, S.: A systematic review of automated test data generation techniques. Masters thesis, School of Engineering., Institute of Technology Box, Ronneby, Sweden (2007)

    Google Scholar 

  27. Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Comput. Surv. 29, 366–427 (1997)

    Google Scholar 

  28. Lipton, R.: Fault diagnosis of computer programs. Technical report, School of Computer Science, Carnegie Mellon University (1971)

    Google Scholar 

  29. Jia, Y., Harman, M.: Java mutation testing repository. UCL (2014). http://crestweb.cs.ucl.ac.uk/resources/mutation_testing_repository/. Cited 28 Sept 2014

  30. Budd, T.A.: Mutation analysis of program test data. Ph.D. thesis, Department of Computer Science, Yale University, New Haven, CT (1980)

    Google Scholar 

  31. DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. Computer 11, 34–41 (1978)

    Article  Google Scholar 

  32. Offutt, A.J., Untch, R.H.: Mutation 2000: uniting the orthogonal. In: Wong, W.E. (ed.) Mutation Testing for the New Century, pp. 34–44. Kluwer Academic Publishers, Norwell (2001)

    Chapter  Google Scholar 

  33. Blum, C., Roli, A.: Metaheuristics in combinatorial optimization: overview and conceptual comparison. ACM Comput. Surv. 35, 268–308 (2003)

    Article  Google Scholar 

  34. Sörensen, K.: Metaheuristics—the metaphor exposed. Int. Trans. Oper. Res. (in press)

    Google Scholar 

  35. Burnstein, I.: Practical Software Testing: A Process-Oriented Approach. Springer, New York (2003)

    MATH  Google Scholar 

  36. Mala, D.J., Mohan, V.: Quality improvement and optimization of test cases: a hybrid genetic algorithm based approach. ACM SIGSOFT Softw. Eng. Notes 35, 1–14 (2010)

    Article  Google Scholar 

  37. Burke, E.K., Kendall, G.: Search methodologies: introductory tutorials in optimization and decision support techniques. Springer, New York (2010)

    MATH  Google Scholar 

  38. Simon, D.: Evolutionary Optimization Algorithms. Wiley, Hoboken (2013)

    Google Scholar 

  39. Forrest, S., Mitchell, M.: Relative building-block fitness and the building-block hypothesis. In: Whitley, D. (ed.) Foundations of Genetic Algorithms 2, pp. 109–126. Morgan Kaufmann, San Mateo (1993)

    Google Scholar 

  40. Harman, M., McMinn, P.: A theoretical and empirical analysis of evolutionary testing and hill climbing for structural test data generation. In: Proceedings of 16th International Symposium Software Testing Analysis, pp. 73–83 (2007)

    Google Scholar 

  41. Arcuri, A.: Full theoretical runtime analysis of alternating variable method on the triangle classification problem. In: Proceedings 1st International Symposium Search Based Software Engineering, pp. 113–121 (2009)

    Google Scholar 

  42. Yoo, S., Harman, M.: Test data regeneration: generating new test data from existing test data. Softw. Test. Verif. Reliab. 22, 171–201 (2012)

    Article  Google Scholar 

  43. Eiben, A.E., Smith, J.E.: Introduction to Evolutionary Computing. Springer, New York (2003)

    Book  MATH  Google Scholar 

  44. Koza, J.R.: Genetic Programming III: Darwinian Invention and Problem Solving. Morgan Kaufmann, San Francisco (1999)

    MATH  Google Scholar 

  45. Patrick, M., Alexander, R., Oriol, M., Clark J.A.: Selecting highly efficient sets of subdomains for mutation adequacy. In: Proceedings 20th Asia-Pacific Software Engineering Conference, pp. 91–98 (2013)

    Google Scholar 

  46. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence: From Natural to Artificial Systems. Oxford University Press, New York (1999)

    MATH  Google Scholar 

  47. Blum, C., Merkle, D.: Swarm Intelligence: Introduction and Applications. Springer, New York (2008)

    Book  MATH  Google Scholar 

  48. Camazine, S., Deneubourg, J.L., Franks, N.R., Sneyd, J., Theraula, G., Bonabeau, E.: Self-organization in Biological Systems. Princeton University Press, Princeton (2003)

    MATH  Google Scholar 

  49. Serugendo, G.M., Gleizes, M.-P., Karageorgos, A.: Self-organising Software: From Natural to Artificial Adaptation. Springer, New York (2011)

    Book  MATH  Google Scholar 

  50. May, P., Timmis, J., Mander, K.: Immune and evolutionary approaches to software mutation testing. Lect. Notes Comput. Sci. 4628, 336–347 (2007)

    Article  Google Scholar 

  51. Passino, K.M.: Biomimicry of bacterial foraging for distributed optimization and control. IEEE Control Syst. 22, 52–67 (2002)

    Article  Google Scholar 

  52. Papadakis, M., Malevris, N.: Automatic mutation test case generation via dynamic symbolic execution. In: Proceedings of 21st International Symposium Software Reliability Engineering, pp. 121–130 (2010)

    Google Scholar 

  53. Harman, M., Yue, J., Langdon, W.B.: Strong higher order mutation-based test data generation. In: Proceedings of 21st ACM SIGSOFT Symposium Foundations Software Engineering, pp. 212–222 (2011)

    Google Scholar 

  54. Chen, T.Y., Kuo, F.-C., Liu, H., Wong, W.E.: Code coverage of adaptive random testing. IEEE Control Syst. 62, 226–237 (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Plant Sciences, University of Cambridge, Downing Street, Cambridge, CB2 3EA, UK

    Matthew Patrick

Authors
  1. Matthew Patrick
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Matthew Patrick .

Editor information

Editors and Affiliations

  1. Department of Electrical and Computer En, University of Alberta, Edmonton, Alberta, Canada

    Witold Pedrycz

  2. Innopolis University, Bolzano, Italy

    Giancarlo Succi

  3. Center for Applied Software Engineering, Bolzano, Italy

    Alberto Sillitti

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Patrick, M. (2016). Metaheuristic Optimisation and Mutation-Driven Test Data Generation. In: Pedrycz, W., Succi, G., Sillitti, A. (eds) Computational Intelligence and Quantitative Software Engineering. Studies in Computational Intelligence, vol 617. Springer, Cham. https://doi.org/10.1007/978-3-319-25964-2_5

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25964-2_5

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25962-8

  • Online ISBN: 978-3-319-25964-2

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics