Skip to main content
SpringerLink
Log in

A generic interoperability testing framework and a systematic development process for automated interoperability testing

  • TTCN-3
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Interoperability is a prerequisite to allow users to access systems implemented by different vendors seamlessly. A good baseline to achieve interoperability is the implementation of a common set of standards. However, this is often not sufficient as different implementations of a standard are not necessarily interoperable. Therefore, different implementations of systems need to be assessed for interoperability by applying interoperability testing. In this article, we present a generic framework that enables automated interoperability testing with message checks, which assess the compliance of messages exchanged between systems. The goal of this framework is the provision of a basic functionality of interoperability test entities, the definition of a generic interoperability test environment, and guidelines for the specification of automated interoperability tests. The framework also considers aspects related to interoperability testing including verdicts, automation, and limitations of the system under test. Through the application of the framework, interoperability of systems can be assessed, systems can be validated, and standards can be improved. In addition, we present a systematic development process for automated interoperability tests to formalize the development and specification of an interoperability test system. We also consider aspects and critical issues, which are important for the development of a complete interoperability test system. The framework and the process are language and system technology independent. We present their application in a case study that includes interoperability tests for the Internet Protocol Multimedia Subsystem (IMS) using the Testing and Test Control Notation Version 3 (TTCN-3).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. Throughout the whole article, we use “standard”, which can be exchanged with “specification” depending on the progress of the standardization.

  2. The TTCN-3 specification of LibIot can be downloaded from http://t3tools.informatik.uni-goettingen.de/trac/browser/trunk/ETSI-Testsuites/ETSI_auto_IOT/ttcn/LibIot.

  3. The IMS interoperability test system can be downloaded from http://t3tools.informatik.uni-goettingen.de/trac/browser/trunk/ETSI-Testsuites/ETSI_auto_IOT.

  4. The TTCN-3 specification of LibIot can be downloaded from http://t3tools.informatik.uni-goettingen.de/trac/browser/trunk/ETSI-Testsuites/ETSI_auto_IOT/ttcn/LibIot.

  5. In the remainder of this article, we use the typewriter font to highlight TTCN-3 identifiers.

  6. The prefix mw is not referring to the Mw interface of IMS but a naming convention indicating a template or message that contains wildcards. Further naming conventions and their explanations can be found in [12].

References

  1. Bergengruen, O., Fischer, F., Namli, T., Rings, T., Schulz, S., Serazio, L., Vassiliou-Gioles, T.: Ensuring interoperability with automated interoperability testing. France, European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2010)

  2. Besse, C., Cavalli, A.R., Kim, M., Zadi, F.: Automated generation of interoperability tests. In: TestCom’02, pp. 169–169 (2002)

  3. Brammer, N.: Investigation of automated terminal interoperability test. Master’s Thesis. Linkpings Universitet, Sweden (2008)

  4. Capgemini, Sogeti, HP: World Quality Report 2011–2012 (2011). [Online; http://www.de.capgemini.com/insights/publikationen/world-quality-report-2011-2012/ fetched on 27–07-2012]

  5. Desmoulin, A., Viho, C.: Automatic interoperability test case generation based on formal definitions. In: Springer LNCS Formal Methods for Industrial Critical Systems, pp. 234–250 (2008)

  6. Dibuz, S., Kremer, P.: Framework and model for automated interoperability test and its application to ROHC. In: TestCom 2003, Springer LNCS 2644, pp. 243–257 (2003)

  7. Dustin, E., Rashka, J., Paul, J.: Automated software testing—introduction, management and performance. Addison-Wesley, USA (1999)

    Google Scholar 

  8. ETSI: 3rd IMS Plugtests™. [Online; http://www.etsi.com/WebSite/OurServices/plugtests/2009IMS3.aspx fetched on 27–07-2012]

  9. ETSI: Plugtests™Interop Events. [Online; http://www.etsi.com/WebSite/OurServices/plugtests/home.aspx fetched on 27–07-2012]

  10. ETSI: ETSI EG 202 810: methods for testing and specification (MTS), automated interoperability testing, methodology and framework. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2010)

  11. ETSI: ETSI ES 202 237: methods for testing and specification (MTS), internet protocol testing (IPT), generic approach to interoperability testing. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2010)

  12. ETSI: ETSI TR 102 788: methods for testing and specification (MTS), automated interoperability testing, specific architectures. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2010)

  13. ETSI: ETSI TR 102 789: methods for testing and specification (MTS), automated interoperability testing, summary of ETSI experiences about using automated interoperability testing tools. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2010)

  14. ETSI: ETSI TS 132 102: digital cellular telecommunications system (Phase 2+), universal mobile telecommunications system (UMTS), LTE, telecommunication management; architecture (3GPP TS 32.102 version 9.0.0 Release 9). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2010)

  15. ETSI: ETSI TS 186 011–2 V2.3.1: IMS network testing (INT), IMS NNI interoperability test specifications, Part 2: test description for IMS NNI interoperability. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2010)

  16. ETSI: ETSI ES 201 873–10: methods for testing and specification (MTS), the testing and test control notation version 3, Part 10: TTCN-3 documentation comment specification. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2011)

  17. ETSI: ETSI ES 201 873: methods for testing and specification (MTS), the testing and test control notation version 3. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2011)

  18. ETSI: ETSI TS 124 229: digital cellular telecommunications system (Phase 2+), universal mobile telecommunications system (UMTS), LTE, IP multimedia call control protocol based on session initiation protocol (SIP) and session description protocol (SDP). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France (2011)

  19. Poikselka, M., Mayer, G.: The IMS: IP multimedia concepts and services. Wiley, New York (2009)

    Google Scholar 

  20. Rings, T., Grabowski, J., Schulz, S.: A testing framework for assessing grid and cloud infrastructure interoperability. Intern. J. Adv. Syst. Measur. (SysMea11v4n12) 3(1—-2), 95–108 (2011)

    Google Scholar 

  21. Schulz, S.: Test suite development with TTCN-3 libraries. Intern. J. Softw. Tools Technol Trans. 10(4), 327–336 (2008)

    Article  Google Scholar 

  22. Seol, S., Kim, M., Kang, S., Ryu, J.: Fully automated interoperability test suite derivation for communication protocols. Comput. Netw. 43(6), 735–759 (2003)

    Article  MATH  Google Scholar 

  23. Spillner, A., Linz, T., Schaefer, H.: Software testing foundations, 2nd edn. Rocky Nook Inc., Santa Barbara (2007)

  24. Vega, D.E., Schieferdecker, I., Din, G.: Design of a test framework for automated interoperability testing of healthcare information systems. In: IEEE Second International Conference on eHealth, Telemedicine, and Social Medicine, pp. 134–140 (2010)

Download references

Acknowledgments

The work carried out here is co-financed by the European Commission within the SA/ETSI/2008-10 ICT Standardisation Programme. The authors would also like to thank the ETSI Technical Committee for Methods for Testing and Specification (TC MTS) and all experts of STF370.

Author information

Authors and Affiliations

  1. Institute of Computer Science, Georg-August-Universität Göttingen, Goldschmidtstr. 7, 37077 , Göttingen, Germany

    Thomas Rings

  2. T-Systems, Berlin, Germany

    Patrick Poglitsch

  3. Conformiq, Espoo, Finland

    Stephan Schulz

  4. Telecom Italia, Turin, Italy

    Luca Serazio

  5. Testing Technologies IST GmbH, Berlin, Germany

    Theofanis Vassiliou-Gioles

Authors
  1. Thomas Rings
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Patrick Poglitsch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stephan Schulz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Luca Serazio
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Theofanis Vassiliou-Gioles
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Thomas Rings.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Rings, T., Poglitsch, P., Schulz, S. et al. A generic interoperability testing framework and a systematic development process for automated interoperability testing. Int J Softw Tools Technol Transfer 16, 295–313 (2014). https://doi.org/10.1007/s10009-013-0281-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-013-0281-2

Keywords

Search

Navigation