Skip to main content
SpringerLink
Log in

Using Model Checking to Generate Fault Detecting Tests

  • Conference paper
Tests and Proofs (TAP 2007)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 4454))

Included in the following conference series:

Abstract

We present a technique which generates from Abstract State Machines specifications a set of test sequences capable to uncover specific fault classes. The notion of test goal is introduced as a state predicate denoting the detection condition for a particular fault. Tests are generated by forcing a model checker to produce counter examples which cover the test goals. We introduce a technique for the evaluation of the fault detection capability of a test set. We report some experimental results which validate the method, compare the fault adequacy criteria with some classical structural coverage criteria and show an empirical cross coverage among faults.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Akers, S.B.: On a theory of boolean functions. Journal Society Industrial Applied Mathematics 7(4), 487–498 (1959)

    Article  MATH  Google Scholar 

  2. Ammann, P., Black, P.E., Ding, W.: Model checkers in software testing. Technical Report NIST-IR 6777, National Institute of Standards and Technology (2002)

    Google Scholar 

  3. Ammann, P., Black, P.E., Majurski, W.: Using model checking to generate tests from specifications. In: ICFEM 1998. 2nd IEEE International Conference on Formal Engineering Methods, Brisbane, Australia, December 1998, p. 46. IEEE Computer Society Press, Los Alamitos (1998)

    Google Scholar 

  4. Ammann, P.E., Black, P.E.: A specification-based coverage metric to evaluate test sets. International Journal of Reliability, Quality and Safety Engineering 8(4), 275–300 (2001)

    Article  Google Scholar 

  5. Atlee, J.M., Buckley, M.A.: A logic-model semantics for SCR software requirements. In: ISSTA 1996. Proceedings of the 1996 ACM SIGSOFT international symposium on Software testing and analysis, pp. 280–292. ACM Press, New York, USA (1996)

    Chapter  Google Scholar 

  6. Barnett, M., Schulte, W.: The ABCs of specification: AsmL, behavior, and components. Informatica 25(4), 517–526 (2001)

    MATH  Google Scholar 

  7. Beyer, D., Chlipala, A.J., Henzinger, T., Jhala, R., Majumdar, R.: Generating tests from counterexamples. In: ICSE 2004. Proc. International Conference on Software Engineering, Edinburgh, May 2004, pp. 326–335. IEEE CS Press, Los Alamitos (2004)

    Google Scholar 

  8. Black, P.E., Okun, V., Yesha, Y.: Mutation of model checker specifications for test generation and evaluation. In: Wong, W.E. (ed.) Mutation Testing for the New Century, proc. of Mutation 2000, October 2000, pp. 14–20. Kluwer Academic Publishers, Dordrecht (2000)

    Google Scholar 

  9. Börger, E., Stärk, R.: Abstract State Machines: A Method for High-Level System Design and Analysis. Springer, Heidelberg (2003)

    MATH  Google Scholar 

  10. Chang, J., Richardson, D.J.: Structural specification-based testing: Automated support and experimental evaluation. In: Nierstrasz, O., Lemoine, M. (eds.) Software Engineering - ESEC/FSE 1999. LNCS, vol. 1687, pp. 285–302. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  11. Chen, T.Y., Lau, M.F.: Test case selection strategies based on boolean specifications. Softw. Test., Verif. Reliab. 11(3), 165–180 (2001)

    Article  Google Scholar 

  12. Chilenski, J., Richey, L.A.: Definition for a masking form of modified condition decision coverage (mcdc). Technical report, Boeing, Seattle WA (1997)

    Google Scholar 

  13. Courtois, P.-J., Parnas, D.L.: Documentation for safety critical software. In: ICSE 1993. Proc. 15th Int’l Conf. on Softw. Eng., Baltimore, MD, pp. 315–323 (1993)

    Google Scholar 

  14. DeMillo, R.A., Guindi, D.S., King, K.N., McCracken, W.M., Offutt, A.J.: An extended overview of the Mothra software testing environment. In: Proceedings of the Second Workshop on Testing, Analysis, and Verification, pp. 142–151. IEEE Computer Society Press, Los Alamitos (1988)

    Chapter  Google Scholar 

  15. Dupuy, A., Leveson, N.: An empirical evaluation of the mc/dc coverage criterion on the hete-2 satellite software. In: The 19th Digital Avionics Systems Conferences. Proceedings DASC (2000)

    Google Scholar 

  16. Engels, A., Feijs, L., Mauw, S.: Test generation for intelligent networks using model checking. In: Brinksma, E. (ed.) TACAS 1997. LNCS, vol. 1217, pp. 384–398. Springer, Heidelberg (1997)

    Chapter  Google Scholar 

  17. Frankl, P.G., Weyuker, E.J.: A formal analysis of the fault-detecting ability of testing methods. IEEE Transactions on Software Engineering 19(3), 202–213 (1993)

    Article  Google Scholar 

  18. Gargantini, A., Heitmeyer, C.: Using model checking to generate tests from requirements specifications. In: Nierstrasz, O., Lemoine, M. (eds.) Software Engineering - ESEC/FSE 1999. LNCS, vol. 1687, pp. 6–10. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  19. Gargantini, A., Riccobene, E.: ASM-based testing: Coverage criteria and automatic test sequence generation. Journal of Universal Computer Science 7(11), 1050–1067 (2001)

    Google Scholar 

  20. Gargantini, A., Riccobene, E., Rinzivillo, S.: Using Spin to generate tests from ASM specifications. In: Börger, E., Gargantini, A., Riccobene, E. (eds.) ASM 2003. LNCS, vol. 2589, pp. 263–277. Springer, Heidelberg (2003)

    Chapter  Google Scholar 

  21. Hamon, G., de Moura, L.M., Rushby, J.M.: Generating efficient test sets with a model checker. In: SEFM 2004. 2nd International Conference on Software Engineering and Formal Methods, Beijing, China, September 28-30, 2004, pp. 261–270 (2004)

    Google Scholar 

  22. Heimdahl, M.P., George, D.: Test-suite reduction for model based tests: Effects on test quality and implications for testing. In: Automated Software Engineering, Linz, Austria (September 2004)

    Google Scholar 

  23. Hierons, R.M.: Comparing test sets and criteria in the presence of test hypotheses and fault domains. ACM Trans. Softw. Eng. Methodol. 11(4), 427–448 (2002)

    Article  Google Scholar 

  24. Holzmann, G.J.: The model checker SPIN. IEEE Transactions on Software Engineering 23(5), 279–295 (1997)

    Article  MathSciNet  Google Scholar 

  25. Hong, H.S., Cha, S.D., Lee, I., Sokolsky, O., Ural, H.: Data flow testing as model checking. In: ICSE 2003, Portland, Oregon, (May 3-10, 2003)

    Google Scholar 

  26. Hong, H.S., Lee, I., Sokolsky, O., Cha, S.D.: Automatic test generation from statecharts using model checking. In: Proceedings of FATES 2001, Workshop on Formal Approaches to Testing of Software, August 2001. BRICS Notes Series, vol. NS-01-4, pp. 15–30 (2001)

    Google Scholar 

  27. IEEE: IEEE Standard Glossary of Software Engineering Terminology. Institute of Electrical and Electronics Engineers, 610.12

    Google Scholar 

  28. Kapoor, K., Bowen, J.P.: Ordering mutants to minimise test effort in mutation testing. In: Grabowski, J., Nielsen, B. (eds.) FATES 2004. LNCS, vol. 3395, pp. 195–209. Springer, Heidelberg (2005)

    Google Scholar 

  29. Kapoor, K., Bowen, J.P.: A formal analysis of MCDC and RCDC test criteria. Softw. Test. Verif. Reliab. 15(1), 21–40 (2005)

    Article  Google Scholar 

  30. Kim, Y.G., Hong, H.S., Cho, S.M., Bae, D.H., Cha, S.D.: Test cases generation from UML state diagrams. IEE Proceedings - Software 146(4), 187–192 (1999)

    Article  Google Scholar 

  31. Kuhn, D.R.: Fault classes and error detection capability of specification-based testing. ACM Transactions on Software Engineering and Methodology 8(4), 411–424 (1999)

    Article  Google Scholar 

  32. Lau, M.F., Yu, Y.-T.: An extended fault class hierarchy for specification-based testing. ACM Trans. Softw. Eng. Methodol. 14(3), 247–276 (2005)

    Article  Google Scholar 

  33. Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines - A survey. Proceedings of The IEEE 84(8), 1090–1123 (1996)

    Article  Google Scholar 

  34. Okun, V., Black, P.E., Yesha, Y.: Comparison of fault classes in specification-based testing. Information and Software Technology 46, 525–533 (2004)

    Article  Google Scholar 

  35. Pretschner, A.: Model-based testing in practice. In: Fitzgerald, J.A., Hayes, I.J., Tarlecki, A. (eds.) FM 2005. LNCS, vol. 3582, pp. 537–541. Springer, Heidelberg (2005)

    Google Scholar 

  36. Rayadurgam, S., Heimdahl, M.P.: Generating MC/DC adequate test sequences through model checking. In: SEW 2003. 28th Annual NASA Goddard Software Engineering Workshop (2003)

    Google Scholar 

  37. Schimd, J.: Executing ASM specifications with AsmGofer, http://www.tydo.de/AsmGofer

  38. Tsuchiya, T., Kikuno, T.: On fault classes and error detection capability of specification-based testing. ACM Trans. Softw. Eng. Methodol. 11(1), 58–62 (2002)

    Article  Google Scholar 

  39. Weyuker, E., Goradia, T., Singh, A.: Automatically generating test data from a Boolean specification. IEEE Transactions on Software Engineering 20(5), 353–363 (1994)

    Article  MATH  Google Scholar 

  40. Zhu, H., Hall, P., May, J.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4), 366–427 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Management and Information Technology, Università di Bergamo,  

    Angelo Gargantini

Authors
  1. Angelo Gargantini
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Yuri Gurevich Bertrand Meyer

Rights and permissions

Reprints and permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Gargantini, A. (2007). Using Model Checking to Generate Fault Detecting Tests. In: Gurevich, Y., Meyer, B. (eds) Tests and Proofs. TAP 2007. Lecture Notes in Computer Science, vol 4454. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-73770-4_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-73770-4_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-73769-8

  • Online ISBN: 978-3-540-73770-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation