Skip to main content
SpringerLink
Log in

Towards a Family of Test Selection Criteria for Symbolic Models of Real-Time Systems

  • Conference paper
  • First Online:
Formal Methods: Foundations and Applications (SBMF 2014)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 8941))

Included in the following conference series:

Abstract

In model-based testing, test cases are generated from a specification model. To avoid an exhaustive search for all possible test cases that can be obtained, usually an expensive and infeasible activity, test case generation may be guided by a test selection criterion. The objective of a test selection criterion is to produce a minimal test suite and yet effective to reveal faults. However, the choice of a criterion is not straightforward specially for real-time systems, because most criteria presented in the literature are general-purpose. Moreover, the relationship between general-purpose and specific criteria for real-time systems is not clear. In this paper, we investigate the criteria that can be applied for test case generation in the scope of model-based testing of real-time systems, specifically of Timed Input-Output Symbolic Transition Systems (TIOSTS) models. We formalize a family of 19 test selection criteria ordered by strict inclusion relation for TIOSTS models. The family combines general-purpose data-flow-oriented and transition-based criteria with specific reactive and real-time systems criteria. We also perform an empirical study to compare the effectiveness of selected criteria. Results of the empirical study indicate that failure detection capability of the generated test suite may vary, but differences are not significant for time failures. We conclude that more effective criteria for the model-based testing of real-time systems are still needed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 34.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 44.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alagar, V.S., Ormandjieva, O., Zheng, M.: Specification-based testing for real-time reactive systems. In: Proceedings of the 34th International Conference on Technology of Object-Oriented Languages and Systems, pp. 25–36 (2000)

    Google Scholar 

  2. Almeida, D.R.: Critérios de Geração de Casos de Teste de Sistemas de Tempo Real. Master’s thesis, Federal University of Campina Grande, Campina Grande, PB, Brazil (2012)

    Google Scholar 

  3. Alur, R., Dill, D.L.: A theory of timed automata. Theoretical Computer Science 126(2), 183–235 (1994)

    Article  MATH  MathSciNet  Google Scholar 

  4. Andrade, W.L., Machado, P.D.L.: Testing interruptions in reactive systems. Formal Aspects of Computing 24, 331–353 (2012)

    Article  MATH  MathSciNet  Google Scholar 

  5. Andrade, W.L., Machado, P.D.L.: Generating test cases for real-time systems based on symbolic models. IEEE Transactions on Software Engineering 39(9), 1216–1229 (2013)

    Article  Google Scholar 

  6. Andrade, W.L., Machado, P.D.L., Jéron, T., Marchand, H.: Abstracting time and data for conformance testing of real-time systems. In: Proceedings of the 8th Workshop on Advances in Model Based Testing, pp. 9–17 (2011)

    Google Scholar 

  7. Arcuri, A., Iqbal, M.Z., Briand, L.: Black-box system testing of real-time embedded systems using random and search-based testing. In: Petrenko, A., Simão, A., Maldonado, J.C. (eds.) ICTSS 2010. LNCS, vol. 6435, pp. 95–110. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  8. Bengtsson, J., Griffioen, W.O.D., Kristoffersen, K.J., Larsen, K.G., Larsson, F., Pettersson, P., Yi, W.: Verification of an audio protocol with bus collision using UPPAAL. In: Alur, R., Henzinger, T.A. (eds.) CAV 1996. LNCS, vol. 1102, pp. 244–256. Springer, Heidelberg (1996)

    Chapter  Google Scholar 

  9. Clarke, D., Lee, I.: Automatic test generation for the analysis of a real-time system: case study. In: Proceedings of the 3rd IEEE Real-Time Technology and Applications Symposium, pp. 112–124 (1997)

    Google Scholar 

  10. El-Far, I.K., Whittaker, J.A.: Model-based software testing. In: Marciniak, J.J. (ed.) Encyclopedia of Software Engineering, vol. 1, pp. 825–837. John Wiley & Sons, Inc. (2002)

    Google Scholar 

  11. En-Nouaary, A., Khendek, F., Dssouli, R.: Fault coverage in testing real-time systems. In: Proceedings of the 6th Real-Time Computing Systems and Applications, pp. 150–157 (1999)

    Google Scholar 

  12. En-Nouaary, A.: Test selection criteria for real-time systems modeled as timed input-output automata. International Journal of Web Information Systems 3(4), 279–292 (2007)

    Article  Google Scholar 

  13. En-Nouaary, A., Hamou-Lhadj, A.: A boundary checking technique for testing real-time systems modeled as timed input output automata. In: Proceedings of the 8th International Conference on Quality Software, pp. 209–215 (2008)

    Google Scholar 

  14. Hessel, A.: Model-Based Test Case Selection and Generation for Real-Time Systems. Ph.D. thesis, Uppsala University, Uppsala, Sweden (2007)

    Google Scholar 

  15. Jeannet, B., Jéron, T., Rusu, V., Zinovieva, E.: Symbolic test selection based on approximate analysis. In: Halbwachs, N., Zuck, L.D. (eds.) TACAS 2005. LNCS, vol. 3440, pp. 349–364. Springer, Heidelberg (2005)

    Chapter  Google Scholar 

  16. Krichen, M., Tripakis, S.: Black-box conformance testing for real-time systems. In: Graf, S., Mounier, L. (eds.) SPIN 2004. LNCS, vol. 2989, pp. 109–126. Springer, Heidelberg (2004)

    Chapter  Google Scholar 

  17. Krichen, M., Tripakis, S.: Conformance testing for real-time systems. Formal Methods in System Design 34(3), 238–304 (2009)

    Article  MATH  Google Scholar 

  18. Laplante, P.A.: Real-Time System Design and Analysis. John Wiley & Sons (2004)

    Google Scholar 

  19. Locke, C.D., Vogel, D.R., Lucas, L., Goodenough, J.B.: Generic avionics software specification. Software Engineering Institute, Carnegie Mellon University, Tech. rep. (1990)

    Google Scholar 

  20. Nielsen, B., Skou, A.: Test generation for time critical systems: tool and case study. In: Proceedings of the 13th Euromicro Conference on Real-Time Systems, pp. 155–162 (2001)

    Google Scholar 

  21. Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: Proceedings of the 8th Workshop on Model-Based Testing, pp. 3–28 (2013)

    Google Scholar 

  22. Pretschner, A., Slotosch, O., Aiglstorfer, E., Kriebel, S.: Model-based testing for real. International Journal on Software Tools for Technology Transfer 5(2), 140–157 (2004)

    Article  Google Scholar 

  23. Rapps, S., Weyuker, E.J.: Selecting software test data using data flow information. IEEE Transactions on Software Engineering 11(4), 367–375 (1985)

    Article  MATH  Google Scholar 

  24. Rusu, V., du Bousquet, L., Jéron, T.: An approach to symbolic test generation. In: Grieskamp, W., Santen, T., Stoddart, B. (eds.) IFM 2000. LNCS, vol. 1945, pp. 338–557. Springer, Heidelberg (2000)

    Chapter  Google Scholar 

  25. Sommerville, I.: Software Engineering. International Computer Science Series, 9th edn. Addison-Wesley, Boston (2010)

    Google Scholar 

  26. Trab, M.S.A., Alrouh, B., Counsell, S., Hierons, R.M., Ghinea, G.: A multi-criteria decision making framework for real time model-based testing. In: Bottaci, L., Fraser, G. (eds.) TAIC PART 2010. LNCS, vol. 6303, pp. 194–197. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  27. Tretmans, J.: Model-based testing and some steps towards test-based modelling. In: Bernardo, M., Issarny, V. (eds.) SFM 2011. LNCS, vol. 6659, pp. 297–326. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  28. Tretmans, J.: Testing concurrent systems: a formal approach. In: Baeten, J.C.M., Mauw, S. (eds.) CONCUR 1999. LNCS, vol. 1664, pp. 46–65. Springer, Heidelberg (1999)

    Chapter  Google Scholar 

  29. Utting, M., Legeard, B.: Practical Model Based Testing: A Tools Approach. Elsevier, San Francisco (2007)

    Google Scholar 

  30. Wohlin, C., Runeson, P., Höst, M., Ohlsson, M.C., Regnell, B., Wesslén, A.: Experimentation in Software Engineering. Springer, New York (2012)

    Book  Google Scholar 

  31. Zheng, M., Alagar, V., Ormandjieva, O.: Automated generation of test suites from formal specifications of real-time reactive systems. Journal of Systems and Software 81(2), 286–304 (2008)

    Article  Google Scholar 

  32. Zhu, H., Hall, P.A.V., May, J.H.R.: Software unit test coverage and adequacy. ACM Computing Surveys 29(4), 366–427 (1997)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. IFPE, Afogados da Ingazeira, PE, Brazil

    Diego R. Almeida

  2. Informatics Center, UFPB, João Pessoa, PB, Brazil

    Alan Moraes

  3. Software Practices Laboratory (SPLab), UFCG, Campina Grande, PB, Brazil

    Alan Moraes, Wilkerson L. Andrade & Patrícia D. L. Machado

Authors
  1. Diego R. Almeida
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Alan Moraes
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wilkerson L. Andrade
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Patrícia D. L. Machado
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alan Moraes .

Editor information

Editors and Affiliations

  1. Universidade Federal Fluminense, Niterói, Brazil

    Christiano Braga

  2. Universidad Complutense de Madrid, Madrid, Spain

    Narciso Martí-Oliet

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Almeida, D.R., Moraes, A., Andrade, W.L., Machado, P.D.L. (2015). Towards a Family of Test Selection Criteria for Symbolic Models of Real-Time Systems. In: Braga, C., Martí-Oliet, N. (eds) Formal Methods: Foundations and Applications. SBMF 2014. Lecture Notes in Computer Science(), vol 8941. Springer, Cham. https://doi.org/10.1007/978-3-319-15075-8_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-15075-8_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-15074-1

  • Online ISBN: 978-3-319-15075-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation