Skip to main content
SpringerLink
Log in

Model-based test generation using extended symbolic grammars

  • TASE 12
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

A novel, model-based test case generation approach for validating reactive systems, especially those supporting richly structured data inputs and/or interactions, is presented. Given an executable system model and an extended symbolic grammar specifying plausible system inputs, the approach performs a model-based simulation to (i) ensure the consistency of the model with respect to the specified inputs, and (ii) generate corresponding test cases for validating the system. The model-based simulation produces a state transition diagram (STD) automatically justifying the model runtime behaviors within the test case coverage. The STD can further be transformed to produce an evolved symbolic grammar, which can then be used to incrementally generate a refined set of test cases. As a case study, we present a live sequence chart (LSC) model-based test generator, named LCT in short, for LSC simulation and consistency testing. The evolved symbolic grammar produced by the simulator can either be used to generate practical test cases for software testing, or be further refined by applying our model-based test generation approach again with additional test coverage criteria. We further show that LSCs can also be used to specify and test certain temporal system properties during the model simulation. Their satisfaction, reflected in the STD, can either be served as a directive for selective test generation, or a basis for further temporal property model checking.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. Internal events are those not starting from objects with waved clouds such as Buyer or Seller.

  2. Hidden events are those related to the structural syntax of LSCs, e.g., end of prechart, or entering a main chart.

  3. A node is called a variant of another if both nodes have the same four-tuple running state except the renaming of symbolic terminals.

  4. Iterative deepening instead of Prolog’s standard depth-first search is used for completeness. Rewriting and heuristics based selection of literals are used to aid termination.

References

  1. Bernot, G., Gaudel, M., Marre, B.: Software testing based on formal specifications: a theory and a tool. Softw. Eng. J. 6(6), 387–405 (1991)

    Article  Google Scholar 

  2. Bertolino, A., Marchetti, E., Muccini, H.: Introducing a reasonably complete and coherent approach for model-based testing. Electron. Notes Theor. Comput. Sci. 116, 85–97 (2005)

    Article  Google Scholar 

  3. Bohn, J., Damm, W., Wittke, H., Klose, J., Moik, A.: Modeling and validating train system applications using statemate and live sequence charts. In: Proceedings of Conference on Integrated Design and Process Technology (IDPT2002), Society for Design and Process Science (2002)

  4. Bontemps, Y., Heymans, P.: Turning high-level live sequence charts into automata. In: Proceedings of Scenarios and State Machines: Models Algorithms and tools, 24th International Conference on Software Engineering, May 2002, ACM (2003)

  5. Bontemps, Y., Heymans, P., Kugler, H.: Applying LSCs to the specification of an air traffic control system. In: Proceedings of Workshop on Scenarios and State Machines: Models, Algorithms and Tools (2003)

  6. Broy, M., Jonsson, B., Katoen, J., Leucker, M., Pretschner, A.: Model-based Testing of Reactive Systems. LNCS(3472), Springer (2005)

  7. Bunker, A., Gopalakrishnan, G., Slind, K.: Live sequence charts applied to hardware requirements specification and verification: a vci bus interface model. Softw. Tools Technol. Transf. 7(4), 250–341 (2005)

    Google Scholar 

  8. Combes, P., Harel, D., Kugler, H.: Modeling and verification of a telecommunication application using live sequence charts and the play-engine tool. In: Peled, D.A., Tsay, Y.-K. (eds.) Automated Technology for Verification and Analysis. Lecture Notes in Computer Science, vol. 3707, pp. 414–428. Springer, Berlin, Heidelberg (2005)

  9. Coppit, D., Lian, J.: Yagg: an easy-to-use generator for structured test inputs. In: Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering. ASE ‘05, pp. 356–359. Long Beach, CA, USA (2005)

  10. Damm, W., Harel, D.: LSCs: breathing life into message sequence charts. In: Proceedings of 3rd IFIP International Conference on Formal Methods for Open Object-based Distributed Systems, pp. 293–312 (1999)

  11. Engels, A., Fiejs, L., Mauw, S.: Test generation for intelligent networks using model checking. In: Proceedings of Workshop on Tools and Algorithms for Construction and Analysis of Systems (TACAS), pp. 384–398 (1997)

  12. Godefroid, P., Kiezun, A., Levin, M.Y.: Grammar-based whitebox fuzzing. In: Proceedings of the ACM SIGPLAN Conference on Programming Languages Design and Implementation (PLDI), pp. 206–215 (2008)

  13. Godefroid, P., Levin, M.Y., Molnar, D.A.: Sage: whitebox fuzzing for security testing. Commun. ACM 55(3), 40–44 (2012)

    Article  Google Scholar 

  14. Goedert, J., Cho, Y., Subramaniam, M., Guo, H.F., Xiao, L.: A framework for virtual interactive construction education (vice). Autom. Constr. 20, 76–87 (2011)

    Article  Google Scholar 

  15. Guo, H.F.: The LCT tool. http://faculty.ist.unomaha.edu/hguo/lct.htm (2014). Accessed 1 Apr 2014

  16. Guo, H.F., Subramaniam, M.: Model-based test generation using evolutional symbolic grammar. In: International Symposium on Theoretical Aspects of Software Engineering (TASE), pp. 111–118 (2012)

  17. Guo, H.F., Zheng, W., Subramaniam, M.: L2C2: logic-based LSC consistency checking. In: 11th International ACM SIGPLAN Symposium on Principles and Practice of Declarative Programming (PPDP) (2009)

  18. Harel, D.: From play-in scenarios to code: an achievable dream. In: Proceedings of Fundamental Approaches to Software Engineering (FASE), pp. 22–34 (2000)

  19. Harel, D., Kugler, H.: Synthesizing state-based object systems from lsc specifications. Int. J. Found. Comput. Sci. 13(1), 5–51 (2002)

    Article  MATH  MathSciNet  Google Scholar 

  20. Harel, D., Maoz, S., Segall, I.: Some results on the expressive power and complexity of LSCs. In: Avron, A., Dershowitz, N., Rabinovich, A. (eds.) Pillars of Computer Science. Lecture Notes in Computer Science, vol. 4800, pp. 351–366. Springer, Berlin, Heidelberg (2008)

  21. Harel, D., Marelly, R.: Come, Let’s Play: Scenario-Based Programming Using LSCs and the Play-Engine. Springer, Berlin, Heidelberg (2003)

  22. Helke, S., Neustupny, T., Santen, T.: Automating test case generation from z specifications using isabelle. In: Proceedings of International Conference of Z Users (ZUM), pp. 52–71 (1997)

  23. Hong, H., Cha, S., Lee, I., Sokolsky, O., Ural, H.: Data flow testing as model checking. In: Proceedings of the International Conference on Software Engineering (ICSE), pp. 232–242 (2003)

  24. Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Proceedings of the 8th International Conference on Tools and Algorithms for the Construction and Analysis of Systems, TACAS ’02, pp. 327–341 (2002)

  25. ITU-T: Message sequence chart (MSC). Z.120 ITU-T recommendation. Available at https://www.itu.int/rec/T-REC-Z.120/en (1996). Accessed 1 Jan 2012

  26. King, J.C.: Symbolic execution and program testing. Commun. ACM 19, 385–394 (1976)

    Article  MATH  Google Scholar 

  27. Klose, J., Toben, T., Westphal, B., Wittke, H.: Check it out: on the efficient formal verification of live sequence charts. In: 18th International Conference on Computer Aided Verification (CAV), pp. 219–233 (2006)

  28. Krenn, W., Schlick, R., Aichernig, B.K.: Mapping UML to labeled transition systems for test-case generation. In: 8th International Symposium on Formal Methods for Components and Objects, pp. 186–207. Springer, Berlin, Heidelberg (2009)

  29. Kumar, R., Mercer, E.: Improving live sequence chart to automata translation for verification. Electron. Commun. EASST 10, 1–14 (2008)

  30. L\(\ddot{\rm a}\)mmel, R., Schulte, W.: Controllable combinatorial coverage in grammar-based testing. In: Testing of Communicating Systems. Lecture Notes in Computer Science, vol. 3964, pp. 19–38. Springer, Berlin, Heidelberg (2006)

  31. Legeard, B., Peureux, F., Utting, M.: Controlling test case explosion in test generation from b formal models. Softw. Test. Verif. Reliab. (STVR) 14(2), 81–103 (2004)

    Article  Google Scholar 

  32. Liu, S., Li, L., Guo, H.F.: Generating test cases via model-based simulation. In: 13th IEEE International Conference on Information Reuse and Integration, pp. 17–24 (2012)

  33. Majumdar, R., Xu, R.G.: Directed test generation using symbolic grammars. In: The ACM SIGSOFT Symposium on the Foundations of Software Engineering: Companion Papers, pp. 553–556 (2007)

  34. Nipkow, T., Paulson, L.C., Wenzel, M.: Isabelle/HOL: a proof assistant for higher-order logic. Lecture Notes in Computer Science, vol. 2283. Springer, Berlin, Heidelberg (2002)

  35. Offutt, J., Liu, S., Abdurazik, A., Ammann, P.: Generating test data from state based specifications. J. Softw. Test. Verif. Reliab. 13, 25–53 (2003)

    Article  Google Scholar 

  36. OMG: Unified modeling languages superstructure specification, v2.0. The Object Management Group. http://www.uml.org/ (2005). Accessed 1 Jan 2012

  37. Pemmasani, G., Guo, H.F., Dong, Y., Ramakrishnan, C., Ramakrishnan, I.: Online justification for tabled logic programs. In: The 7th International Symposium on Functional and Logic Programming, pp. 24–38 (2004)

  38. Rothermel, G., Harrold, M.J.: A safe, efficient regression test selection technique. ACM Trans. Softw. Eng. Methodol. 6, 173–210 (1997)

    Article  Google Scholar 

  39. Rothermel, G., Harrold, M.J., von Ronne, J., Hong, C.: Empirical studies of test suite reduction. J. Softw. Test. Verif. Reliab. 4, 219–249 (2002)

    Article  Google Scholar 

  40. Sirer, E.G., Bershad, B.N.: Using production grammars in software testing. In: 2nd Conference on Domain Specific Languages, pp. 1–13 (1999)

  41. Sun, J., Dong, J.S.: Model checking live sequence charts. In: The 10th IEEE International Conference on Engineering of Complex Computer Systems, pp. 529–538 (2005)

  42. Tan, L., Sokolsky, O., Lee, I.: Specification-based testing with linear temporal logic. In: Proceedings of IEEE International Conference on Information Reuse and Integration (IRI’04), pp. 493–498 (2004)

  43. Toben, T., Westphal, B.: On the expressive power of LSCs. In: The 32nd Conference on Current Trends in Theory and Practice of Computer Science, pp. 33–43 (2006)

  44. Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan-Kaufmann, San Francisco, CA, USA (2007)

  45. Warren, D.S.: Memoing for logic programs. Commun. ACM 35, 93–111 (1992)

    Article  MathSciNet  Google Scholar 

  46. Zheng, W.: Consistency checking on LSC specifications. Master Thesis, University of Nebraska at Omaha, Omaha (2009)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Nebraska at Omaha, Omaha, NE, 68182, USA

    Hai-Feng Guo & Mahadevan Subramaniam

Authors
  1. Hai-Feng Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mahadevan Subramaniam
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Hai-Feng Guo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Guo, HF., Subramaniam, M. Model-based test generation using extended symbolic grammars. Int J Softw Tools Technol Transfer 16, 437–455 (2014). https://doi.org/10.1007/s10009-014-0316-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-014-0316-3

Keywords

Search

Navigation