Skip to main content
Springer Nature Link
Log in

On the Relationship Between the Order of Mutation Testing and the Properties of Generated Higher Order Mutants

  • Conference paper
Intelligent Information and Database Systems (ACIIDS 2016)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 9621))

Included in the following conference series:

  • 2433 Accesses

Abstract

The goal of higher order mutation testing is to improve mutation testing effectiveness in particular and test effectiveness in general. There are different approaches which have been proposed in the area of second order mutation testing and higher order mutation testing with mutants order ranging from 2 to 70. Unfortunately, the empirical evidence on the relationship between the order of mutation testing and the desired properties of generated mutants is scarce except the conviction that the number of generated mutants could grow exponentially with the order of mutation testing. In this paper, we present the study of finding the relationships between the order of mutation testing and the properties of mutants in terms of number of generated high quality and reasonable mutants as well as generated live mutants. Our approach includes higher order mutants classification, objective functions and fitness functions to classify and identify generated higher order mutants. We use four multi-objective optimization algorithms for constructing higher order mutants. Obtained empirical results indicate that 5 is a relevant highest order in higher order mutation testing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

Notes

  1. 1.

    http://www.mutationtesting.org/.

References

  1. DeMillo, R.A., Lipton, R.J., Sayward, F.G.: Hints on test data selection: help for the practicing programmer. IEEE Comput. 11(4), 34–41 (1978)

    Article  Google Scholar 

  2. Hamlet, R.G.: Testing programs with the aid of a compiler. IEEE Trans. Software Eng. 3(4), 279–290 (1977)

    Article  MathSciNet  MATH  Google Scholar 

  3. Harman, M., Jia, Y., Langdon, W.B.: A manifesto for higher order mutation testing. In: Proceedings of the 2010 Third International Conference on Software Testing, Verification, and Validation Workshops. ICSTW 2010, pp. 80–89. IEEE Computer Society, Washington, DC (2010)

    Google Scholar 

  4. Jia, Y., Harman, M.: Constructing subtle faults using higher order mutation testing. In: Source Code Analysis and Manipulation. pp. 249–258 (2008)

    Google Scholar 

  5. Jia, Y., Harman, M.: Higher order mutation testing. Inf. Softw. Technol. 51(10), 1379–1393 (2009)

    Article  Google Scholar 

  6. Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Softw. Eng. 37(5), 649–678 (2011)

    Article  Google Scholar 

  7. Kintis, M., Papadakis, M., Malevris, N.: Evaluating mutation testing alternatives: a collateral experiment. In: 2010 17th Asia Pacific Software Engineering Conference (APSEC), pp. 300–309, November 2010

    Google Scholar 

  8. Langdon, W.B., Harman, M., Jia, Y.: Efficient multi-objective higher order mutation testing with genetic programming. J. Syst. Softw. 83(12), 2416–2430 (2010)

    Article  Google Scholar 

  9. Madeyski, L.: On the effects of pair programming on thoroughness and fault-finding effectiveness of unit tests. In: Münch, J., Abrahamsson, P. (eds.) PROFES 2007. LNCS, vol. 4589, pp. 207–221. Springer, Heidelberg (2007). doi:10.1007/978-3-540-73460-4_20

    Chapter  Google Scholar 

  10. Madeyski, L.: Impact of pair programming on thoroughness and fault detection effectiveness of unit test suites. Softw. Process Improv. Pract. 13(3), 281–295 (2008). doi:10.1002/spip.382. http://madeyski.e-informatyka.pl/download/Madeyski08.pdf

    Article  Google Scholar 

  11. Madeyski, L.: The impact of test-first programming on branch coverage and mutation score indicator of unit tests: an experiment. Inf. Softw. Technol. 52(2), 169–184 (2010). doi:10.1016/j.infsof.2009.08.007

    Article  Google Scholar 

  12. Madeyski, L., Orzeszyna, W., Torkar, R., Józala, M.: Overcoming the equivalent mutant problem: a systematic literature review and a comparative experiment of second order mutation. IEEE Trans. Softw. Eng. 40(1), 23–42 (2014). doi:10.1109/TSE.2013.44

    Article  Google Scholar 

  13. Madeyski, L., Radyk, N.: Judy - a mutation testing tool for java. IET Softw. 4(1), 32–42 (2010). doi:10.1049/iet-sen.2008.0038

    Article  Google Scholar 

  14. Nguyen, Q.V., Madeyski, L.: Problems of mutation testing and higher order mutation testing. In: van Do, T., Thi, H.A.L., Nguyen, N.T. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 282, pp. 157–172. Springer, Heidelberg (2014). doi:10.1007/978-3-319-06569-4_12

    Chapter  Google Scholar 

  15. Nguyen, Q.V., Madeyski, L.: Searching for strongly subsuming higher order mutants by applying multi-objective optimization algorithm. In: Le Thi, H.A., Nguyen, N.T., Do, T.V. (eds.) Advanced Computational Methods for Knowledge Engineering. AISC, vol. 358, pp. 391–402. Springer, Heidelberg (2015). doi:10.1007/978-3-319-17996-4_35

    Google Scholar 

  16. Nguyen, Q.V., Madeyski, L.: Empirical evaluation of multi-objective optimization algorithms searching for higher order mutants. An International Journal on Cybernetics andSystems (2016). doi:10.1080/01969722.2016.1128763, http://madeyski.e-informatyka.pl/download/NguyenMadeyski16CS.pdf

    Google Scholar 

  17. Omar, E., Ghosh, S., Whitley, D.: Constructing subtle higher order mutants for Java and aspectJ programs. In: 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE), pp. 340–349, November 2013

    Google Scholar 

  18. Papadakis, M., Malevris, N.: An empirical evaluation of the first and second order mutation testing strategies. In: 2010 Third International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), pp. 90–99, April 2010

    Google Scholar 

  19. Polo, M., Piattini, M., García-Rodríguez, I.: Decreasing the cost of mutation testing with second-order mutants. Softw. Test. Verification Reliab. 19(2), 111–131 (2009). doi:10.1002/stvr.392

    Article  Google Scholar 

  20. Purushothaman, R., Perry, D.E.: Toward understanding the rhetoric of small source code changes. IEEE Trans. Softw. Eng. 31(6), 511–526 (2005)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Science and Management, Wroclaw University of Technology, Wroclaw, Poland

    Quang Vu Nguyen & Lech Madeyski

Authors
  1. Quang Vu Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lech Madeyski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Quang Vu Nguyen .

Editor information

Editors and Affiliations

  1. Wrocław University of Technology, Wrocław, Poland

    Ngoc Thanh Nguyen

  2. Wrocław University of Technology, Wrocław, Poland

    Bogdan Trawiński

  3. Iwate Prefectural University, Takizawa, Japan

    Hamido Fujita

  4. National University of Kaohsiung, Kaohsiung, Taiwan

    Tzung-Pei Hong

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nguyen, Q.V., Madeyski, L. (2016). On the Relationship Between the Order of Mutation Testing and the Properties of Generated Higher Order Mutants. In: Nguyen, N.T., Trawiński, B., Fujita, H., Hong, TP. (eds) Intelligent Information and Database Systems. ACIIDS 2016. Lecture Notes in Computer Science(), vol 9621. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-49381-6_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-49381-6_24

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-49380-9

  • Online ISBN: 978-3-662-49381-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics