Abstract
We propose a novel framework for model-based testing against specifications from EULYNX, a SysML-based standard from the railway industry for the controllers of systems such as points, signals, sensors, and crossings. The main challenge here is the sheer complexity: with state spaces exceeding \(10^{10}\) states, it is hard to derive test suites that achieve a meaningful type of coverage.
We tackle this problem by moving away from the traditional interleaving semantics for SysML. Instead, we propose a synchronous semantics in terms of Finite State Machines (FSMs), leveraging the fact that EULYNX is implemented on Programmable Logic Controllers (PLCs). Then, we deploy Single-Input-Change Deterministic Finite State Machines (SIC-DFSMs), which ensures fully deterministic tests thus minimizing scalability issues.
Our focus lies on the EULYNX specification for point controllers. The generated test suite achieves maximal transition coverage, but test execution time remains substantial. We introduce an additional test suite that achieves maximal transition label coverage. Remarkably, this smaller suite successfully identifies the same four faults as the larger suite.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Change history
20 March 2024
A correction has been published.
References
Bachmann, T., van der Wal, D., van der Bijl, M., van der Meij, D., Oprescu, A.: Translating EULYNX SysML models into symbolic transition systems for model-based testing of railway signaling systems. 2022 IEEE Conference on Software Testing, Verification and Validation (ICST), pp. 355–364 (2022)
Basile, D., et al.: On the industrial uptake of formal methods in the railway domain. In: Furia, C., Winter, K. (eds.) IFM 2018. LNCS, vol. 11023, pp. 20–29. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-98938-9_2
Bonacchi, A., Fantechi, A., Bacherini, S., Tempestini, M.: Validation process for railway interlocking systems. Sci. Comput. Program. 128, 2–21 (2016)
Bouwman, M., van der Wal, D., Luttik, B., Stoelinga, M., Rensink, A.: A case in point: verification and testing of a EULYNX interface. Formal Aspects Comput. 35, 1–38 (2022)
Braunstein, C., et al.: Complete model-based equivalence class testing for the ETCS ceiling speed monitor. In: Merz, S., Pang, J. (eds.) ICFEM 2014. LNCS, vol. 8829, pp. 380–395. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-11737-9_25
Bunte, O., et al.: The mCRL2 toolset for analysing concurrent systems - improvements in expressivity and usability. In: International Conference on Tools and Algorithms for Construction and Analysis of Systems (2019)
EULYNX website. http://eulynx.eu. Accessed 18 Jan 2023
Fantechi, A.: Twenty-five years of formal methods and railways: what next? In: SEFM Workshops (2013)
Ferrari, A., ter Beek, M.H.: Formal methods in railways: a systematic mapping study. ACM Comput. Surv. 55, 1–37 (2022). https://doi.org/10.1145/3520480
Gay, G., Staats, M., Whalen, M.W., Heimdahl, M.P.E.: The risks of coverage-directed test case generation. IEEE Trans. Software Eng. 41, 803–819 (2015)
Graf-Brill, A., Hermanns, H.: Model-based testing for asynchronous systems. In: FMICS-AVoCS (2017)
Haxthausen, A.E., Peleska, J.: model checking and model-based testing in the railway domain. In: SyDe Summer School (2015)
Huo, J., Petrenko, A.: Transition covering tests for systems with queues. Softw. Testing 19, 55–83 (2009)
International Electrotechnical Commission: International Standard IEC 61131: Programmable Controllers (2017)
Jia, Y., Harman, M.: An analysis and survey of the development of mutation testing. IEEE Trans. Software Eng. 37, 649–678 (2011)
Kadakolmath, L., Ramu, U.D.: Model-checking-based automated test case generation for Z formal specification of an urban railway interlocking system. In: 2022 Fourth International Conference on Emerging Research in Electronics, Computer Science and Technology (ICERECT), pp. 1–8 (2022)
Kanellakis, P.C., Smolka, S.A.: CCS expressions, finite state processes, and three problems of equivalence. Inf. Comput. 86, 43–68 (1983)
Khan, S.U.R., Lee, S.P., Javaid, N., Abdul, W.: A systematic review on test suite reduction: approaches, experiment’s quality evaluation, and guidelines. IEEE Access 6, 11816–11841 (2018)
Kiran, A., Butt, W.H., Anwar, M.W., Azam, F., Maqbool, B.: A comprehensive investigation of modern test suite optimization trends, Tools and Techniques. IEEE Access 7, 89093–89117 (2019)
Lee, D., Yannakakis, M.: Principles and methods of testing finite state machines-a survey. Proc. IEEE 84, 1090–1123 (1996)
Liu, S., et al.: A formal semantics for complete UML state machines with communications. In: Johnsen, E.B., Petre, L. (eds.) IFM 2013. LNCS, vol. 7940, pp. 331–346. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-38613-8_23
Lv, J., Wang, H., Liu, H., Zhang, L., Tang, T.: A model-based test case generation method for function testing of train control systems. In: 2016 IEEE International Conference on Intelligent Rail Transportation (ICIRT), pp. 334–346 (2016)
Ma, C., Jordan, C.V., Provost, J.: SATE: model-based testing with design-to-test and plant features. IFAC-PapersOnLine 51, 310–315 (2018)
Ma, C., Provost, J.: Design-to-test: an approach to enhance testability of programmable controllers for critical systems-two case studies (2016)
Ma, C., Provost, J.: Design-to-test approach for programmable controllers in safety-critical automation systems. IEEE Trans. Industr. Inf. 16, 6499–6508 (2020)
Noroozi, N., Khosravi, R., Mousavi, M.R., Willemse, T.A.C.: Synchronizing asynchronous conformance testing. In: Barthe, G., Pardo, A., Schneider, G. (eds.) SEFM 2011. LNCS, vol. 7041, pp. 334–349. Springer, Heidelberg (2011). https://doi.org/10.1007/978-3-642-24690-6_23
Object Management Group: OMG Unified Modeling Language, Version 2.5.1 (2017). https://www.omg.org/spec/UML/
Object Management Group: OMG Systems Modeling Language, Version 1.6 (2019). https://www.omg.org/spec/SysML/
Paltor, I.: The Semantics of UML State Machines (1999)
Peleska, J.: Industrial-strength model-based testing - state of the art and current challenges. In: MBT (2013)
Polze, A.: EULYNX-Live: a methodology for validating system specifications in hybrid field tests EULYNX-Live: Eine Methodik zum Validieren von Systemspezifikationen in hybriden Feldtests (2021)
Provost, J., Roussel, J.M., Faure, J.M.: Testing programmable logic controllers from finite state machines specification. In: 2011 3rd International Workshop on Dependable Control of Discrete Systems, pp. 1–6 (2011)
Provost, J., Roussel, J.M., Faure, J.M.: Generation of single input change test sequences for conformance test of programmable logic controllers. IEEE Trans. Industr. Inf. 10, 1696–1704 (2014)
Salunkhe, S., Berglehner, R., Rasheeq, A.: Automatic transformation of SysML model to event-B model for railway CCS application. In: International Conference on Abstract State Machines, Alloy, B, TLA, VDM, and Z (2021)
Sánchez, C., Cavalli, A.R., Yevtushenko, N.V., Santos, J., Abreu, R.: On modeling and testing components of the European train control system (2014)
Scippacercola, F., Pietrantuono, R., Russo, S., Zentai, A.: Model-in-the-loop testing of a railway interlocking system. In: Desfray, P., Filipe, J., Hammoudi, S., Pires, L.F. (eds.) MODELSWARD 2015. CCIS, vol. 580, pp. 375–389. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27869-8_22
Sehr, M.A., et al.: programmable logic controllers in the context of industry 4.0. IEEE Trans. Industr. Inf. 17, 3523–3533 (2021)
Su, H., Chai, M., Liu, H., Chai, J., Yue, C.: A model-based testing system for safety of railway interlocking. 2022 IEEE 25th International Conference on Intelligent Transportation Systems (ITSC), pp. 335–340 (2022)
Tretmans, J.: Model based testing with labelled transition systems. In: Formal Methods and Testing (2008)
Utting, M., Pretschner, A., Legeard, B.: A taxonomy of model-based testing approaches. Softw. Testing 22, 297–312 (2012)
Verhaard, L., Tretmans, J., Kars, P., Brinksma, E.: On asynchronous testing. In: Protocol Test Systems (1992)
Virazel, A., David, R., Girard, P., Landrault, C., Pravossoudovitch, S.: Delay fault testing: choosing between random SIC and random MIC test sequences. J. Electron. Test. 17, 233–241 (2000)
Wang, Y., Chen, L., Kirkwood, D., Fu, P., Lv, J., Roberts, C.: Hybrid online model-based testing for communication-based train control systems. IEEE Intell. Transp. Syst. Mag. 10, 35–47 (2018)
Yi, W., Xing-hua, F., Dai-qiang, W.: An implementation of random single input change technique for low-power test. In: 2008 2nd International Conference on Anti-counterfeiting, Security and Identification, pp. 352–355 (2008)
Acknowledgements
This paper is a product of the FormaSig project, fully funded by DB Netz AG and ProRail. The vision illustrated in this article reflects the personal views of the authors, and is not part of the strategy of DB Netz AG or ProRail. We thank the SIGNON Group (https://signon-group.com/) for providing access to the source code of the software simulator.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
van der Wal, D., Gerhold, M., Stoelinga, M. (2023). Conformance in the Railway Industry: Single-Input-Change Testing a EULYNX Controller. In: Cimatti, A., Titolo, L. (eds) Formal Methods for Industrial Critical Systems. FMICS 2023. Lecture Notes in Computer Science, vol 14290. Springer, Cham. https://doi.org/10.1007/978-3-031-43681-9_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-43681-9_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-43680-2
Online ISBN: 978-3-031-43681-9
eBook Packages: Computer ScienceComputer Science (R0)