Skip to main content
SpringerLink
Log in

Back-To-Back Testing of Model-Based Code Generators

  • Conference paper
Book cover Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change (ISoLA 2014)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 8802))

Abstract

In this paper, we present the testing approach of the Genesys code generator framework. The employed approach is based on back-to-back-testing, which tests the translation performed by a code generator from a semantic perspective rather than just checking for syntactic correctness of the generation result. We describe the basic testing framework and show that it scales in three dimensions: parameterized tests, testing across multiple target platforms and testing on multiple meta-levels.

In particular, the latter is only possible due to the fact that Genesys code generators are constructed as models. Furthermore, in order to facilitate simplicity, Genesys consistently employs one single notation for all artifacts involved in this testing approach: Test data, test cases, the code generators under test, and even the testing framework itself are all modeled using the same graphical modeling language.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Bakera, M., Margaria, T., Renner, C.D., Steffen, B.: Tool-supported enhancement of diagnosis in model-driven verification. Innovations in Systems and Software Engineering 5(3), 211–228 (2009)

    Article  Google Scholar 

  2. Bazzichi, F., Spadafora, I.: An automatic generator for compiler testing. IEEE Transactions on Software Engineering SE 8(4), 343–353 (1982)

    Article  MATH  Google Scholar 

  3. Beck, K.: Test Driven Development: By Example. Addison-Wesley (2002)

    Google Scholar 

  4. Boujarwah, A.S., Saleh, K.: Compiler test case generation methods: a survey and assessment. Information & Software Technology 39(9), 617–625 (1997)

    Article  Google Scholar 

  5. Celentano, A., Crespi-Reghizzi, S., Vigna, P.D., Ghezzi, C., Granata, G., Savoretti, F.: Compiler testing using a sentence generator. Softw., Pract. Exper. 10(11), 897–918 (1980)

    Article  Google Scholar 

  6. Clarke, E.M., Grumberg, O., Peled, D.: Model Checking. MIT Press (1999)

    Google Scholar 

  7. Czarnecki, K., Helsen, S.: Feature-based survey of model transformation approaches. IBM Systems Journal 45, 621–645 (2006)

    Article  Google Scholar 

  8. Hagerer, A., Hungar, H., Margaria, T., Niese, O., Steffen, B., Ide, H.-D.: Demonstration of an Operational Procedure for the Model-Based Testing of CTI Systems. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 336–340. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  9. Hagerer, A., Hungar, H.: Model Generation by Moderated Regular Extrapolation. In: Kutsche, R.-D., Weber, H. (eds.) FASE 2002. LNCS, vol. 2306, pp. 80–95. Springer, Heidelberg (2002)

    Chapter  Google Scholar 

  10. Hoare, C.A.R.: An axiomatic basis for computer programming. Communications of the ACM 12, 576–580 (1969)

    Article  MATH  Google Scholar 

  11. Homer, W., Schooler, R.: Independent testing of compiler phases using a test case generator. Softw., Pract. Exper. 19(1), 53–62 (1989)

    Article  Google Scholar 

  12. Howar, F., Steffen, B., Jonsson, B., Cassel, S.: Inferring canonical register automata. In: Kuncak, V., Rybalchenko, A. (eds.) VMCAI 2012. LNCS, vol. 7148, pp. 251–266. Springer, Heidelberg (2012)

    Chapter  Google Scholar 

  13. Howar, F., Steffen, B., Merten, M.: Automata learning with automated alphabet abstraction refinement. In: Jhala, R., Schmidt, D. (eds.) VMCAI 2011. LNCS, vol. 6538, pp. 263–277. Springer, Heidelberg (2011)

    Chapter  Google Scholar 

  14. Isberner, M., Howar, F., Steffen, B.: Learning register automata: from languages to program structures. Machine Learning, 1–34 (2013)

    Google Scholar 

  15. Jörges, S.: Construction and Evolution of Code Generators. LNCS, vol. 7747. Springer, Heidelberg (2013)

    Google Scholar 

  16. Jörges, S., Lamprecht, A.L., Margaria, T., Schaefer, I., Steffen, B.: A constraint-based variability modeling framework. STTT 14(5), 511–530 (2012)

    Article  Google Scholar 

  17. Jörges, S., Steffen, B.: Exploiting ecore’s reflexivity for bootstrapping domain-specific code-generators. In: 35th Annual IEEE Software Engineering Workshop, SEW 2012, pp. 72–81. IEEE Computer Society (2012)

    Google Scholar 

  18. Kahn, G.: Natural Semantics. In: Brandenburg, F.J., Vidal-Naquet, G., Wirsing, M. (eds.) STACS 1987. LNCS, vol. 247, pp. 22–39. Springer, Heidelberg (1987)

    Google Scholar 

  19. Kleppe, A.: Software Language Engineering: Creating Domain-Specific Languages Using Metamodels, 1st edn. Addison-Wesley (2008)

    Google Scholar 

  20. Leroy, X.: Formal certification of a compiler back-end or: programming a compiler with a proof assistant. In: POPL 2006, pp. 42–54. ACM (2006)

    Google Scholar 

  21. Margaria, T., Steffen, B.: Service Engineering: Linking Business and IT. IEEE Computer 39(10), 45–55 (2006)

    Article  Google Scholar 

  22. Margaria, T., Steffen, B.: Agile it: Thinking in user-centric models. In: Margaria, T., Steffen, B. (eds.) ISoLA 2008. CCIS, vol. 17, pp. 490–502. Springer, Heidelberg (2008)

    Google Scholar 

  23. Margaria, T., Steffen, B.: Simplicity as a driver for agile innovation. IEEE Computer 43(6), 90–92 (2010)

    Article  Google Scholar 

  24. Naujokat, S., Neubauer, J., Lamprecht, A.L., Steffen, B., Jörges, S., Margaria, T.: Simplicity-first model-based plug-in development. Softw., Pract. Exper. 44(3), 277–297 (2014)

    Article  Google Scholar 

  25. Necula, G.C.: Proof-carrying code. In: Proceedings of the 24th ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages, POPL 1997, pp. 106–119. ACM (1997)

    Google Scholar 

  26. Niese, O., Steffen, B., Margaria, T., Hagerer, A., Brune, G., Ide, H.-D.: Library-Based Design and Consistency Checking of System-Level Industrial Test Cases. In: Hussmann, H. (ed.) FASE 2001. LNCS, vol. 2029, pp. 233–248. Springer, Heidelberg (2001)

    Chapter  Google Scholar 

  27. Parr, T.: Enforcing strict model-view separation in template engines. In: Proceedings of the 13th International Conference on World Wide Web, WWW 2004, pp. 224–233. ACM (2004)

    Google Scholar 

  28. Plotkin, G.D.: A Structural Approach to Operational Semantics. Tech. Rep. DAIMI FN–19, Computer Science Department, Aarhus University (1981)

    Google Scholar 

  29. Pnueli, A., Siegel, M.D., Singerman, E.: Translation Validation. In: Steffen, B. (ed.) TACAS 1998. LNCS, vol. 1384, pp. 151–166. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  30. Raffelt, H., Margaria, T., Steffen, B., Merten, M.: Hybrid test of web applications with webtest. In: Proceedings of the 2008 Workshop on Testing, analysis, and Verification of Web Services and Applications, TAV-WEB 2008, pp. 1–7. ACM (2008)

    Google Scholar 

  31. Raffelt, H., Merten, M., Steffen, B., Margaria, T.: Dynamic testing via automata learning. International Journal on Software Tools for Technology Transfer (STTT) 11(4), 307–324 (2009)

    Article  Google Scholar 

  32. Sampath, P., Rajeev, A.C., Ramesh, S., Shashidhar, K.C.: Behaviour directed testing of auto-code generators. In: Sixth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2008), pp. 191–200. IEEE Computer Society (2008)

    Google Scholar 

  33. Schmidt, D.A.: Denotational semantics: a methodology for language development. William C. Brown Publishers (1986)

    Google Scholar 

  34. Stahl, T., Völter, M., Efftinge, S., Haase, A.: Modellgetriebene Softwareentwicklung: Techniken, Engineering, Management. dpunkt, 2nd edn. (2007) (in German)

    Google Scholar 

  35. Steffen, B., Margaria, T., Nagel, R., Jörges, S., Kubczak, C.: Model-Driven Development with the jABC. In: Bin, E., Ziv, A., Ur, S. (eds.) HVC 2006. LNCS, vol. 4383, pp. 92–108. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  36. Stürmer, I., Conrad, M.: Code Generator Testing in Practice. In: INFORMATIK 2004 - Informatik verbindet, Band 2, Beiträge der 34. Jahrestagung der Gesellschaft für Informatik e.V (GI), pp. 33–37. GI (2004)

    Google Scholar 

  37. Stürmer, I., Conrad, M., Doerr, H., Pepper, P.: Systematic Testing of Model-Based Code Generators. IEEE Transactions on Software Engineering 33, 622–634 (2007)

    Article  Google Scholar 

  38. Stürmer, I., Weinberg, D., Conrad, M.: Overview of existing safeguarding techniques for automatically generated code. In: Proceedings of SEAS 2005, pp. 1–6. ACM (2005)

    Google Scholar 

  39. Tillmann, N., Schulte, W.: Parameterized unit tests. In: Proceedings of the 10th European Software Engineering Conference, pp. 253–262. ACM (2005)

    Google Scholar 

  40. Vouk, M.A.: Back-to-back testing. Information and Software Technology 32, 34–45 (1990)

    Article  Google Scholar 

  41. Windmüller, S., Neubauer, J., Steffen, B., Howar, F., Bauer, O.: Active continuous quality control. In: Proceedings of the 16th ACM SIGSOFT Symposium on Component Based Software Engineering (CBSE), pp. 111–120. ACM (2013)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Programming Systems, Technische Universität Dortmund, Germany

    Sven Jörges & Bernhard Steffen

Authors
  1. Sven Jörges
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Bernhard Steffen
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. University of Limerick, Ireland

    Tiziana Margaria

  2. TU Dortmund, Germany

    Bernhard Steffen

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Jörges, S., Steffen, B. (2014). Back-To-Back Testing of Model-Based Code Generators. In: Margaria, T., Steffen, B. (eds) Leveraging Applications of Formal Methods, Verification and Validation. Technologies for Mastering Change. ISoLA 2014. Lecture Notes in Computer Science, vol 8802. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-45234-9_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-45234-9_30

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-45233-2

  • Online ISBN: 978-3-662-45234-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation