Skip to main content
SpringerLink
Log in

Quantifying the evolution of TTCN-3 as a language

  • TTCN-3
  • Published:
International Journal on Software Tools for Technology Transfer Aims and scope Submit manuscript

Abstract

Ten years of maintenance, nine published revisions of the standards for the Testing and Test Control Notation version 3 (TTCN-3), more than 500 change requests since 2006, and 10 years of activity on the official TTCN-3 mailing list add up to a rich history, not unlike that of many successful Open Source Software (OSS) projects. In this article, we contemplate TTCN-3 in the context of software evolution and examine its history quantitatively. We mined the changes in the textual content of the standards, the data in change requests from the past 5 years, and the mailing list archives from the past 10 years. In addition, to characterize the use of the TTCN-3 we investigated the meta-data of the contributions at the TTCN-3 User Conference, and the use of language constructs in a large-scale TTCN-3 test suite. Based on these data sets, we first analyze the amount, density, and location of changes within the different parts of the standard. Then, we analyze the activity and focus of the user community and the maintenance team in both the change request management system and the official TTCN-3 mailing list. Finally, we analyze the distribution of contributions at the TTCN-3 User Conference across different topics over the past 8 years and construct use anomalies during the development of a large-scale test suite. Our findings indicate that the TTCN-3 is becoming increasingly stable as the overall change density and intensity, as well as the number of change requests are decreasing, despite the monotonous increase in the size of the standards.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27

Similar content being viewed by others

Notes

  1. In the first revision, Part 3 was published as a technical report rather than a full standard.

  2. Coloring schemes can be applied for specific purposes, such as highlighting particular properties of individual changes.

  3. Data was last collected on 01.11.2011, with the last activity recorded at 14:55 on 01.11.2011.

  4. Note that (1) there are also interim revisions included in the figure that have not been officially published, where change requests implemented in these are also integrated in the subsequent officially published revision, and (2) change requests for the next revision of the standard (v4.4.1) are also included, as it is already being worked on.

  5. While in [38] the last session in 2009 is scheduled for 23–27.11.2009, upon further investigation, we found out that this session had been rescheduled and took place 2 weeks later, 07-11.12.2009.

  6. A single change requests may reference more than one clause.

  7. Data was last collected on 14.10.2011, with the last message dated 28.09.2011.

  8. We did check if this was a coincidence with the same title recurring by accident in two distinct discussions. It was not the case. It was indeed a single response to a 5-year-old question. The author excused themselves for not looking at the date of the original question shortly thereafter in a follow-up message.

  9. Note that data regarding the maintenance team members is missing for 2003 and 2006. We estimated that the messages by the maintenance team in these periods follow the overall trends.

  10. AltstepDef and AltstepInstance are plotted against the secondary vertical scale on the right-hand side.

  11. Guard statements in TTCN-3 provide means to set conditions for the eligibility of alternative branches within an alt statement or an altstep.

References

  1. HOPL-III: Proceedings of the Third ACM SIGPLAN Conference on History of Programming Languages. ACM, New York (2007)

  2. Chen, H., Lynch, K.J.: Automatic construction of networks of concepts characterizing document databases. IEEE Trans. Syst. Man Cybern. 22(5), 885–902 (1992). doi:10.1109/21.179830

    Article  Google Scholar 

  3. Cook, W.R.: AppleScript. pp. 1–1-1-21. ACM Press (2007) doi:10.1145/1238844.1238845. http://dl.acm.org/citation.cfm?doid=1238844.1238845

  4. ETSI: Technical Report (TR) 101 666 (1999–05): Information technology—open systems interconnection conformance testing methodology and framework, the tree and tabular combined notation (TTCN) (Ed. 2++). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (1999)

  5. ETSI: ETSI Standard (ES) 201 873–2 V3.2.1: The Testing and Test Control Notation Version 3, Part 2: TTCN-3 Tabular Presentation Format (TFT). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2007) (also published as ITU-T Recommendation Z.162)

  6. ETSI: ETSI Standard (ES) 201 873–3 V3.2.1: The Testing and Test Control Notation Version 3, Part 3: TTCN-3 Graphical Presentation Format (TFT). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2007) (also published as ITU-T Recommendation Z.163)

  7. ETSI: ETSI Standard (ES) 201 873–4 V4.2.1: The Testing and Test Control Notation Version 3, Part 4: TTCN-3 Operational Semantics. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2010) also published as ITU-T Recommendation Z.164

  8. ETSI: ETSI Standard (ES) 202 781 V1.1.1: TTCN-3 Extensions—Configuration and Deployment Support. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2010)

  9. ETSI: ETSI Standard (ES) 202 782 V1.1.1: TTCN-3 Extensions—Performance and Real Time Testing. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2010)

  10. ETSI: ETSI Standard (ES) 201 873–1 V4.3.1: The Testing and Test Control Notation Version 3, Part 1: TTCN-3 Core Language. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.161

  11. ETSI: ETSI Standard (ES) 201 873–10 V4.3.1: TTCN-3 Documentation Comment Specification. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.170

  12. ETSI: ETSI Standard (ES) 201 873–5 V4.3.1: The Testing and Test Control Notation Version 3, Part 5: TTCN-3 Runtime Interface (TRI). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.165

  13. ETSI: ETSI Standard (ES) 201 873–6 V4.3.1: The Testing and Test Control Notation Version 3, Part 6: TTCN-3 Control Interface (TCI). European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.166

  14. ETSI: ETSI Standard (ES) 201 873–7 V4.3.1: The Testing and Test Control Notation Version 3, Part 7: TTCN-3 Using ASN.1 with TTCN-3. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.167

  15. ETSI: ETSI Standard (ES) 201 873–8 V4.3.1: The Testing and Test Control Notation Version 3, Part 8: TTCN-3 the IDL to TTCN-3 Mapping. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.168

  16. ETSI: ETSI Standard (ES) 201 873–9 V4.3.1: The Testing and Test Control Notation Version 3, Part 9: TTCN-3 Using XML Schema with TTCN-3. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011) also published as ITU-T Recommendation Z.169

  17. ETSI: ETSI Standard (ES) 202 784 V1.2.1: TTCN-3 Extensions—Advanced Parameterization. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011)

  18. ETSI: ETSI Standard (ES) 202 785 V1.2.1: TTCN-3 Extensions—Behaviour Types. European Telecommunications Standards Institute (ETSI), Sophia-Antipolis (2011)

  19. ETSI: TTCN-3 change request submission rules. http://www.ttcn-3.org/ChangeRequest.htm (2011)

  20. ETSI: TTCN-3 change requests. http://t-ort.etsi.org/view_all_bug_page.php (2011)

  21. ETSI: TTCN-3 mailing list archives. http://list.etsi.org/archives/ttcn3.html (2011)

  22. ETSI: TTCN-3 maintenance team working schedules. http://t-ort.etsi.org/main_page.php (2011)

  23. Fischer, M., Pinzger, M., Gall, H.: Populating a release history database from version control and bug tracking systems. In: International Conference on Software Maintenance, 2003. ICSM 2003. Proceedings, pp. 23–32. IEEE (2003). doi:10.1109/ICSM.2003.1235403

  24. Grabowski, J., Hogrefe, D., Rethy, G., Schieferdecker, I., Wiles, A., Willcock, C.: An introduction to the testing and test control notation (TTCN-3). Comput. Netw. Intern. J. Comput. Telecommun. Netw. 42(3), 375–403 (2003). doi:10.1016/S1389-1286(03)00249-4

    MATH  Google Scholar 

  25. Griswold, R.E., Griswold, M.T.: History of the Icon Programming Language. pp. 53–68. ACM Press (1993). doi:10.1145/154766.155363. http://portal.acm.org/citation.cfm?doid=154766.155363

  26. Hudak, P., Hughes, J., Jones, S.P., Wadler, P.: A History of Haskell. pp. 12–1-12-55. ACM Press (2007). doi:10.1145/1238844.1238856. http://dl.acm.org/citation.cfm?doid=1238844.1238856

  27. Ierusalimschy, R., de Figueiredo, L.H., Celes, W.: The Evolution of Lua. pp. 2–1-2-26. ACM Press (2007). doi:10.1145/1238844.1238846. http://dl.acm.org/citation.cfm?doid=1238844.1238846

  28. ISO/IEC: International standard ISO/IEC 9646–3:1998: Information technology—open systems interconnection-conformance testing methodology and framework-Part 3: the tree and tabular combined notation (TTCN). International Organization for Standardization/International Electrotechnical Commission, France (1992)

  29. ISO/IEC: International standard ISO/IEC 9646–3:1998: Information technology—open systems interconnection-conformance testing methodology and framework-Part 3: the tree and tabular combined notation (TTCN). International Organization for Standardization/International Electrotechnical Commission, UK (1998)

  30. ITEA: Itea web site of the TT-Medal project-tests and testing methodologies with advanced languages. http://www.itea2.org/project/index/view/?project=53 (2011)

  31. L-Soft: LISTSERV email list management software. http://www.lsoft.com/products/listserv.asp (2011)

  32. Levenshtein, V.I.: Binary codes capable of correcting deletions, insertions, and reversals. Sov. Phy. Doklady 10(8), 707–710 (1966)

    MathSciNet  Google Scholar 

  33. MantisBT Group: Mantis bug tracker. http://www.mantisbt.org/ (2011)

  34. Monkewich, O.: TTCN, past, present and future. In: Presentation at the TTCN-3 User Conference 2011, Bled, Slovenia, 7–9 June (2011)

  35. Nagappan, N., Ball, T.: Use of relative code churn measures to predict system defect density. In: 27th International Conference on Software Engineering, 2005. ICSE 2005. Proceedings, pp. 284–292. IEEE (2005). doi:10.1109/ICSE.2005.1553571

  36. NLM: National library of medicine lexical tools (2011). http://lexsrv3.nlm.nih.gov/LexSysGroup/Projects/lvg/2011/web/index.html

  37. Parnin, C., Bird, C., Murphy-Hill, E.: Java generics adoption: how new features are introduced, championed, or ignored. In: Proceeding of the 8th Working Conference on Mining Software Repositories, MSR ’11, p. 312. ACM, New York (2011). doi:10.1145/1985441.1985446

  38. Rethy, G.: The new TTCN-3 version 4.1.1. In: Presentation at the TTCN-3 User Conference 2009, Sophia-Antipolis, 3–5 June (2009)

  39. Standard, E.S.S.: EBNF: ISO/IEC 14977: 1996 (E). http://www.cl.cam.ac.uk/mgk25/iso-14977.pdf (1996)

  40. Stroustrup, B.: Evolving a language in and for the real world. In: Proceedings of the Third ACM SIGPLAN Conference on History of Programming Languages-HOPL III, pp. 4.1–4.59. San Diego, (2007). doi:10.1145/1238844.1238848. http://dl.acm.org/citation.cfm?id=1238844.1238848&coll=DL&dl=ACM&CFID=39540141&CFTOKEN=55983406

  41. TRex Team: TRex Website (2007). http://www.trex.informatik.uni-goettingen.de

  42. Willcock, C., Deiß, T., Tobies, S., Keil, S., Engler, F., Schulz, S.: An Introduction to TTCN-3, 2nd edn (2011). http://onlinelibrary.wiley.com/book/10.1002/9780470977903

  43. Zeiß, B.: A refactoring ool for TTCN-3. Master’s thesis, Masterarbeit im Studiengang Angewandte Informatik am Institut für Informatik, ZFI-BM-2006-05, ISSN, pp. 1612–6793 (2006)

Download references

Author information

Authors and Affiliations

  1. Software Engineering for Distributed Systems Group, Institute of Computer Science, University of Göttingen, Goldschmidtstr. 7, 37077 , Göttingen, Germany

    Philip Makedonski, Jens Grabowski & Florian Philipp

Authors
  1. Philip Makedonski
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jens Grabowski
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Florian Philipp
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philip Makedonski.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Makedonski, P., Grabowski, J. & Philipp, F. Quantifying the evolution of TTCN-3 as a language. Int J Softw Tools Technol Transfer 16, 227–246 (2014). https://doi.org/10.1007/s10009-013-0282-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10009-013-0282-1

Keywords

Search

Navigation