Abstract
In this talk I will discuss the combination of model-based testing and mutation testing. Model-based testing is a black-box testing technique that avoids the labour of manually writing hundreds of test cases, but instead advocates the capturing of the expected behaviour in a model of the system under test. The test cases are automatically generated from this model. The technique is receiving growing interest in the embedded-systems domain, where models are the rule rather than the exception.
Mutation testing is a technique for assessing and improving a test suite. A number of faulty versions of a program under test are produced by injecting bugs into its source code. These faulty programs are called mutants. A tester analyses if his test suite can ”kill” all mutants. We say that a test kills a mutant if it is able to distinguish it from the original. The tester improves his test suite until all faulty mutants get killed.
In model-based mutation testing, we combine the central ideas of model-based testing and mutation testing: we inject bugs in a model and generate a test suite that will kill these bugs. In this talk, I will discuss its scientific foundations, tools, and results. The foundations include semantics and conformance relations; the supporting tools involve model checkers, constraint solvers and SMT solvers; our experimental results are taken from two European projects on embedded-systems. I will conclude with a proposal how model-based mutation testing can be integrated into an agile, iterative development process.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Aichernig, B.K.: Mutation Testing in the Refinement Calculus. Formal Aspects of Computing 15(2-3), 280–295 (2003)
Aichernig, B.K., Brandl, H., Jöbstl, E., Krenn, W.: Efficient mutation killers in action. In: IEEE Fourth International Conference on Software Testing, Verification and Validation, ICST 2011, Berlin, Germany, March 21-25, pp. 120–129. IEEE Computer Society (2011)
Aichernig, B.K., Delgado, C.C.: From Faults Via Test Purposes to Test Cases: On the Fault-Based Testing of Concurrent Systems. In: Baresi, L., Heckel, R. (eds.) FASE 2006. LNCS, vol. 3922, pp. 324–338. Springer, Heidelberg (2006)
Aichernig, B.K., He, J.: Mutation testing in UTP. Formal Aspects of Computing 21(1-2), 33–64 (2009)
Aichernig, B.K., Jöbstl, E.: Efficient refinement checking for model-based mutation testing. In: Proceedings of the 12th International Conference on Quality Software (QSIC 2012). IEEE Computer Society (in press, 2012)
Aichernig, B.K., Jöbstl, E.: Towards symbolic model-based mutation testing: Combining reachability and refinement checking. In: 7th Workshop on Model-Based Testing (MBT 2012). EPTCS, vol. 80, pp. 88–102 (2012)
Aichernig, B.K., Jöbstl, E.: Towards symbolic model-based mutation testing: Pitfalls in expressing semantics as constraints. In: Workshops Proc. of the 5th Int. Conf. on Software Testing, Verification and Validation (ICST 2012), pp. 752–757. IEEE Computer Society (2012)
Aichernig, B.K., Salas, P.A.P.: Test case generation by OCL mutation and constraint solving. In: Cai, K.-Y., Ohnishi, A. (eds.) Fifth International Conference on Quality Software, QSIC 2005, Melbourne, Australia, September 19-21, pp. 64–71. IEEE Computer Society (2005)
Arbab, F.: Reo: A Channel-based Coordination Model for Component Composition. Mathematical Structures in Computer Science 14(3), 329–366 (2004)
Back, R.-J., Kurki-Suonio, R.: Decentralization of process nets with centralized control. In: 2nd ACM SIGACT-SIGOPS Symposium on Principles of Distributed Computing, PODC 1983, pp. 131–142. ACM (1983)
Brandl, H., Weiglhofer, M., Aichernig, B.K.: Automated conformance verification of hybrid systems. In: Wang, J., Chan, W.K., Kuo, F.-C. (eds.) Proceedings of the 10th International Conference on Quality Software, QSIC 2010, Zhangjiajie, China, July 14-15, pp. 3–12. IEEE Computer Society (2010)
Gries, D.: The Science of Programming. Texts and Monographs in Computer Science. Springer (1981)
Jard, C., Jéron, T.: TGV: theory, principles and algorithms. International Journal on Software Tools for Technology Transfer (STTT) 7(4), 297–315 (2005)
Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Transactions on Software Engineering 37(5), 649–678 (2011)
Krenn, W., Aichernig, B.K.: Test case generation by contract mutation in Spec#. In: Finkbeiner, B., Gurevich, Y., Petrenko, A.K. (eds.) Proceedings of Fifth Workshop on Model Based Testing (MBT 2009), York, England, March 22. Electronic Notes in Theoretical Computer Science, vol. 253(2), pp. 71–86. Elsevier (October 2009)
Krenn, W., Schlick, R., Aichernig, B.K.: Mapping UML to Labeled Transition Systems for Test-Case Generation – A Translation via Object-Oriented Action Systems. In: de Boer, F.S., Bonsangue, M.M., Hallerstede, S., Leuschel, M. (eds.) FMCO 2009. LNCS, vol. 6286, pp. 186–207. Springer, Heidelberg (2010)
Meng, S., Arbab, F., Aichernig, B.K., Astefanoaei, L., de Boer, F.S., Rutten, J.: Connectors as designs: Modeling, refinement and test case generation. Science of Computer Programming 77(7-8), 799–822 (2012)
Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann Publishers (2007)
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Software Testing, Verification and Reliability (2011)
Weiglhofer, M., Aichernig, B.K., Wotawa, F.: Fault-based conformance testing in practice. International Journal of Software and Informatics 3(2-3), 375–411 (2009); Special double issue on Formal Methods of Program Development edited by Dines Bjoerner
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Aichernig, B.K. (2012). The Science of Killing Bugs in a Black Box. In: de Carvalho Junior, F.H., Barbosa, L.S. (eds) Programming Languages. Lecture Notes in Computer Science, vol 7554. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-33182-4_2
Download citation
DOI: https://doi.org/10.1007/978-3-642-33182-4_2
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-33181-7
Online ISBN: 978-3-642-33182-4
eBook Packages: Computer ScienceComputer Science (R0)