Skip to main content
SpringerLink
Log in

An IMS Performance Benchmark Implementation based on the TTCN-3 Language

  • Special section on advances in test automation – the evolution of TTCN-3
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Performance benchmarking is a method to assess performance characteristics of various systems across different system architectures under realistic conditions. A benchmark is executed to collect measurements such as fail rate, response times or round-trip delays. Additionally, it can be used to identify scalability or usability problems of the system under heavy load. This paper describes a performance benchmark for the IP Multimedia Subsystem (IMS). IMS is a standardised architecture for providing a unified session control on top of various access network technologies realising flexible multimedia applications. In the context of IMS, where a scalability of up to several millions of subscribers is required, the goal of a benchmarking system is to help understand how a system will perform when the number of calls per second and the number of active users increase. The Testing and Test Control Notation (TTCN-3) has been used to implement the benchmark specification.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Abbreviations

CSCF:

Call Session Control Functions

DOC:

Design Objective Capacity

ETSI:

European Telecommunication Standards Institute

I-CSCF:

Interrogating Call Session Control Functions

CSCF:

Call Session Control Functions

DOC:

Design Objective Capacity

ETSI:

European Telecommunication Standards Institute

I-CSCF:

Interrogating Call Session Control Functions

IHS:

Inadequately Handled Scenarios

IHSA:

Inadequately Handled Scenario Attempts

IMS:

IP Multimedia Subsystem

IP:

Internet Protocol

NIST:

National Institute of Standards and Technology

NGN:

Next Generation Networking

P-CSCF:

Proxy Call Session Control Functions

QoS:

Quality of Service

S-CSCF:

Serving Call Session Control Functions

SIP:

Session Initiation Protocol

SLF:

Subscriber Location Function

SUT:

System Under Test

TCI:

TTCN-3 Control Interfaces

TLI:

TTCN-3 Logging Interface

TRI:

TTCN-3 Runtime Interface

TS:

Test System

TTCN-3:

Testing and Test Control Notation

UPSF:

User Profile Server Function

VoIP:

Voice over IP

3GPP:

3rd Generation Partnership Project

References

  1. 3GPP (2006) Technical specification group services and system aspects. IP Multimedia Subsystem (IMS), Stage 2, V5.15.0, TS 23.228

  2. Barber, S. (2004) Creating Effective Load Models for Performance Testing with Incomplete Empirical Data. wse 00:51–59, ISSN 1550–4441

  3. Dibuz, S., Szabó, T., Torpis, Z.: BCMP performance test with TTCN-3 mobile node emulator. In: Groz, R., Hierons. R.M. (eds.) TestCom. Lecture Notes in Computer Science, vol. 2978, pp. 50–59, ISBN 3-540-21219-1 (2004)

  4. Din, G., Tolea, S., Schieferdecker, I.: Distributed load tests with TTCN-3. In: Uyar, M.Ü., Duale, A.Y., Fecko, M.A. (Eds.) TestCom. Springer, Lecture Notes in Computer Science, vol. 3964, pp. 177–196, ISBN 3-540-34184-6 (2006)

  5. ETSI (2005) ETSI Technical Specification TS 102 027-3: SIP ATS & PIXIT; Part 3: Abstract Test Suite (ATS) and partial Protocol Implementation eXtra Information for Testing (PIXIT). Sophia-Antipolis, France

  6. ETSI (2007a) ETSI European Standard (ES) 201 873-1 V3.2.1 (2007-02) The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language. Sophia-Antipolis, France

  7. ETSI (2007b) ETSI European Standard (ES) 201 873-5 V3.2.1 (2007-02) The Testing and Test Control Notation version 3; Part 5: TTCN-3 Runtime Interface (TRI). Sophia-Antipolis, France

  8. ETSI (2007) ETSI European Standard (ES) 201 873-6 V3.2.1 (2007-02): The Testing and Test Control Notation version 3; Part 6: TTCN-3 Control Interface (TCI). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis, France

  9. ETSI TISPAN (2007) IMS/NGN Performance Benchmark, Technical Standard (TS) 186 008. Sophia-Antipolis, France

  10. Fraunhofer FOKUS (2006) FOKUS Open Source IMS Core. http://www.openimscore.org

  11. Fraunhofer FOKUS (2006–2007) IMS Benchmarking Project. http://www.fokus.fraunhofer.de/IMSBenchmarking?lang=en

  12. Gao, J.Z., Tsao, J., Wu, Y., Jacob, T.H.-S.: Testing and Quality Assurance for Component-Based Software. Artech House, Inc., Norwood, MA, USA, ISBN 1580534805 (2003)

  13. Gayraud, R., Jaques, O.: SIPp (2006) http://sipp.sourceforge.net

  14. Grabowski J., Hogrefe D., Rthy G., Schieferdecker I., Wiles A., Willcock C.: An introduction into the testing and test control notation (TTCN-3). Comp. Netw. 42(3), 375–403 (2003)

    Article  MATH  Google Scholar 

  15. IETF (2005) RFC 3261, SIP: Session Initiation Protocol. http://tools.ietf.org/html/rfc3261

  16. Jain R.: The art of computer systems performance analysis: techniques for experimental design, measurement, simulation and modelling. Wiley- Interscience, New York (1991)

    Google Scholar 

  17. Menasce, D., Almeida, V., Fonseca, R., Mendes, M.A.: A methodology for workload characterization of E-commerce sites. ACM Press, Denver, pp. 119–128, ISBN 1-58113-176-3 (1999)

  18. Network Working Group (2005) RFC 4180, Common Format and MIME Type for Comma-Separated Values (CSV) Files. http://tools.ietf.org/html/rfc4180

  19. NIST/SEMATECH (2006) e-Handbook of Statistical Methods. http://www.itl.nist.gov/div898/handbook

  20. Petre, R.: Load balancing and parallelization strategies within a TTCN-3 test system. Diploma thesis, University Politechnica Bucharest (2006)

  21. Ranganathan, M., O’Doherty, P.: JAIN SIP Tutorial (2005) http://java.sun.com/products/jain/JAIN-SIP-Tutorial.pdf

  22. Schieferdecker, I., Din, G., Apostolidis, D.: Distributed functional and load tests for Web services. Int. J. Softw. Tools Technol. Transf. 7(4), 351–360, ISSN 1433-2779 (2005)

    Google Scholar 

  23. Schieferdecker, I., Stepien, B., Rennoch, A.: PerfTTCN, a TTCN language extension for performance testing. In: IFIP TC6 10th International Workshop on Testing of Communicating Systems (IWTCS’97) Proceedings, Chapman & Hall, London, pp. 21–36 (1997)

  24. Schulzrinne, H., Narayanan, S., Lennox, J., Doyle, M.: SipStone Benchmarking SIP Server Performance. http://www.sipstone.org/files/sipstone_0402.pdf, Columbia University (2006)

  25. Smith, C., Williams, L.: Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software, 1st edn. Addison-Wesley Professional, ISBN 0201722291 (2001)

  26. Splaine, S., Jaskiel, S.: The Web Testing Handbook. S T Q E Pub, ISBN 0970436300 (2001)

  27. TestingTechnologies (2006) TTworkbench: an Eclipse based TTCN-3 IDE. http://www.testingtech.de/products/ttwb_intro.php

Download references

Author information

Authors and Affiliations

  1. Fraunhofer FOKUS, Kaiserin-Augusta-Allee 31, 10589, Berlin, Germany

    George Din

Authors
  1. George Din
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to George Din.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Din, G. An IMS Performance Benchmark Implementation based on the TTCN-3 Language. Int J Softw Tools Technol Transf 10, 359–370 (2008). https://doi.org/10.1007/s10009-008-0078-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-008-0078-x

Keywords

Search

Navigation