Skip to main content
SpringerLink
Log in

Test descriptions with ETSI TDL

  • Published:
Software Quality Journal Aims and scope Submit manuscript

Abstract

To address the need for abstract, high-level test descriptions that can be shared among different stakeholders, the European Telecommunications Standards Institute (ETSI) commissioned the design of the Test Description Language (TDL). TDL is designed as a domain-specific language for testing, consisting of a standardised abstract syntax (meta-model) and concrete syntaxes for textual specification, graphical design, and model exchange between tools. Its main purpose is to support a test methodology that is followed in the standardisation work for software-intense systems at ETSI and is applicable in industrial projects as well. TDL enables the formal specification of both test objectives derived from system requirements and test descriptions refining the test objectives. The latter serve as blueprint for the implementation of executable tests. A standardised mapping of TDL specifications to test scripts in the standardised test execution language TTCN-3 widens the reach of TDL to ensure compatibility and consistency of generated executable tests. An open-source toolset has been developed as a common platform to accelerate the adoption of TDL and lower the barrier to entry for users and tool vendors. Reports from pilot applications within three ETSI standardisation groups demonstrate the practicality of the chosen approach.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20

Similar content being viewed by others

Notes

  1. http://www.etsi.org

  2. https://www.eclipse.org/sirius/

  3. https://eclipse.org/Xtext/

  4. http://www.docx4java.org

  5. https://www.eclipse.org/papyrus/

  6. http://tdl.etsi.org and http://top.etsi.org

  7. http://www.etsi.org/about/how-we-work/testing-and-interoperability/centre-for-testing-and-interoperability

  8. http://www.etsi.org/technologies-clusters/technologies/nfv

  9. See ETSI NFV 003, http://www.etsi.org/deliver/etsi_gs/NFV/001_099/003/01.02.01_60/gs_NFV003v010201p.pdf.

  10. http://www.etsi.org/deliver/etsi_gr/NFV-TST/001_099/007/01.01.01_60/gr_NFV-TST007v010101p.pdf

  11. http://www.onem2m.org

  12. http://www.3gpp.org

  13. http://www.etsi.org/deliver/etsi_ts/136500_136599/13652301/14.02.00_60/ts_13652301v140200p.pdf

  14. https://robotframework.org

References

  • ASAM ATX (2012). Automotive test exchange format (ATX) v1.0.0. ASAM standard. https://www.asam.net/standards/detail/atx/.

  • Bouquet, F, Grandpierre, C, Legeard, B, Peureux, F, Vacelet, N, Utting, M. (2007). A subset of precise uml for model-based testing. In Proceedings of the 3rd Int’l workshop on advances in model-based testing (A-MOST’07) (pp. 95–104). New York: ACM.

  • ETSI EG 203 130 (2013). Methods for Testing and Specification (MTS); Model-Based Testing (MBT); methodology for standardised test specification development, v1.1.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 201 873-1 (2016). Methods for Testing and Specification (MTS); the testing and test control notation version 3; Part 1: core language, v4.8.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 202 553 (2009). Methods for Testing and Specification (MTS); TPLan: a notation for expressing test purposes, v1.2.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 202 782 (2015). Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 extensions: performance and real-time testing, v1.3.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 202 786 (2017). Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; TTCN-3 Extensions: support of interfaces with continuous signals, v1.4.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 202 951 (2011). Methods for Testing and Specification (MTS); Model-Based Testing (MBT); requirements for modelling notations, v1.1.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-1 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 1: abstract syntax and associated semantics, v1.4.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-2 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 2: graphical syntax, v1.3.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-3 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 3: exchange format, v1.3.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-4 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 4: structured test objective specification (extension), v1.3.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-5 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 5: UML profile for TDL, v1.1.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-6 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 6: mapping to TTCN-3, v1.1.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI ES 203 119-7 (2018). Methods for Testing and Specification (MTS); The Test Description Language (TDL); Part 7: extended test configurations, v1.1.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • ETSI TR 102 840 (2011). Methods for Testing and Specification (MTS); Model-Based Testing (MBT); Model-Based Testing in Standardisation, v1.2.1 European Telecommunications Standards Institute (ETSI). Sophia-Antipolis, France.

  • Grossmann, J, & Müller, W. (2006). A formal behavioral semantics for testML. In Second international symposium on leveraging applications of formal methods, verification and validation, 2006. ISoLA 2006 (pp. 441-448).

  • Guduvan, A, Waeselynck, H, Wiels, V, Durrieu, G, Fusero, Y, Schieber, M. (2013). A meta-model for tests of avionics embedded systems. In Proc. of the 1st Int’l conference on model-driven engineering and software development (MODELSWARD) (pp. 5–13). Barcelona.

  • Hammond, S, & Umphress, D. (2012). Test driven development: the state of the practice. In Proc. of the 50th annual southeast regional conference (pp. 158–163). ACM.

  • ISO 13209 (2011). Road vehicles – Open Test sequence eXchange format (OTX) – Parts 1–3. International ISO multipart standard No. 13209-1 to 3. https://www.iso.org/ics/43.020/x/.

  • ISO/IEC 9646-1 (1994). Information technology – open systems interconnection – conformance testing methodology and framework – Part 1: general concepts. International ISO/IEC multipart standard No. 9646-1. https://www.iso.org/standard/17473.html.

  • ISO/IEC/IEEE 29119-3 (2013). ISO/IEC/IEEE international standard – software and systems engineering – software testing – part 3: test documentation. Tech. rep., ISO/IEC/IEEE. https://ieeexplore.ieee.org/servlet/opac?punumber=6588538.

  • ITU-T Z120 (2011). Series Z: languages and general software aspect for telecommunication systems – formal description techniques (FDT) – message sequence chart (MSC). International Telecommunication Union, Recommendation Z.120 (02/11), document http://www.itu.int/rec/T-REC-Z.120-201102-I/en.

  • ITU-T Z120 Annex B (1998). Formal semantics of message sequence charts, 04/98. International telecommunication union, recommendation Z.120 Annex B (04/98), document http://www.itu.int/rec/T-REC-Z.120-199804-I!AnnB/en.

  • Kaner, C. (2003). On scenario testing. STQE Magazine, 16–22.

  • Kanstrén, T., Puolitaival, O.P, Rytky, V.M., Saarela, A., Keränen, J.S. (2012). Experiences in setting up domain-specific model-based testing. In 2012 IEEE International conference on industrial technology (ICIT) (pp. 319–324).

  • Kolovos, D.S., Paige, R.F., Polack, F.A.C. (2009). On the evolution of OCL for capturing structural constraints in modelling languages. In Rigorous methods for software construction and analysis. Essays Dedicated to Egon Börger on the Occasion of His 60th Birthday (pp. 204– 218).

  • Makedonski, P, Adamis, G, Käärik, M, Kristoffersen, F, Zeitoun, X. (2016). Evolving the ETSI test description language. In Proc. of the 9th system analysis and modelling conference (SAM 2016). Springer.

  • Micskei, Z, & Waeselynck, H. (2010). The many meanings of UML 2 Sequence Diagrams: A survey. Software & Systems Modeling, 10(4), 489–514.

    Article  Google Scholar 

  • OMG MARTE. (2011). UML profile for MARTE: modeling and analysis of real-time embedded systems, version 1.1. object management group, document number: formal/2011-06-02, document http://www.omg.org/spec/MARTE/1.1/.

  • OMG MOF (2014). Meta object facility core, version 2.4.2. Object management group, document number: formal/2014-04-05, document http://www.omg.org/spec/MOF/2.4.2/.

  • OMG OCL (2012). Object constraint language, version 2.3.1. Object management group, document number: formal/2012-05-09, document http://www.omg.org/spec/OCL/2.3.1/.

  • OMG UML (2015). Unified modeling language, version 2.5, chapter 12.3 (profiles). Object management group, document number: formal/2015-03-01, document http://www.omg.org/spec/UML/2.5/.

  • OMG UTP (2013). UML testing profile, version 1.2. Object management group, document number: formal/2013-04-03, document http://www.omg.org/spec/UTP/1.2/.

  • OMG XMI (2014). XML metadata interchange, version 2.4.2. Object management group, document number: formal/2014-04-06, document http://www.omg.org/spec/XMI/2.5.1/.

  • Pajunen, T, Takala, T, Katara, M. (2011). Model-based testing with a general purpose keyword-driven test automation framework. In 4th IEEE Int’l conference on software testing, verification and validation, ICST 2012 (pp. 242–251). Berlin: Workshop Proceedings.

  • Razorcat (2014). CCDL whitepaper. Razorcat Technical Report, 23 January 2014. https://www.razorcat.com/de/downloads-ccdl.html.

  • Solis, C, & Wang, X. (2011). A study of the characteristics of behaviour driven development. In 2011 37th EUROMICRO Conference on software engineering and advanced applications (SEAA) (pp. 383–387). IEEE.

  • Ulrich, A, Jell, S, Votintseva, A, Kull, A. (2014). The ETSI test description language TDL and its application. In 2nd Int’l conference on model-driven engineering and software development (MODELSWARD) (pp. 601–608).

Download references

Funding

The work on Test Description Language (TDL) has been partially funded by the European Telecommunications Standards Institute (ETSI) in the context of the Specialist Task Force (STF) projects 454, 476, 492, and 522 between 2013 and 2017.

Author information

Authors and Affiliations

  1. Institute of Computer Science, University of Göttingen, Göttingen, Germany

    Philip Makedonski & Jens Grabowski

  2. Test Competence Center, Ericsson Hungary Ltd., Budapest, Hungary

    Gusztáv Adamis

  3. Elvior OU, Tallinn, Estonia

    Martti Käärik

  4. Cinderella ApS, Copenhagen, Denmark

    Finn Kristoffersen

  5. Centre for Testing and Interoperability, European Telecommunications Standards Institute, Sophia-Antipolis, France

    Michele Carignani

  6. Corporate Technology, Siemens AG, Munich, Germany

    Andreas Ulrich

Authors
  1. Philip Makedonski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Gusztáv Adamis
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martti Käärik
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Finn Kristoffersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Michele Carignani
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Andreas Ulrich
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jens Grabowski
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philip Makedonski.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Makedonski, P., Adamis, G., Käärik, M. et al. Test descriptions with ETSI TDL. Software Qual J 27, 885–917 (2019). https://doi.org/10.1007/s11219-018-9423-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11219-018-9423-9

Keywords

Search

Navigation