Skip to main content
SpringerLink
Log in

Model-based testing of software for automation systems using heuristics and coverage criterion

  • Theme Section Paper
  • Published:
Software & Systems Modeling Aims and scope Submit manuscript

Abstract

The aim of this work is to increase the confidence on software for automation systems defining a coverage criterion to measure the quality level of generated tests and the time interval needed to execute them. This coverage criterion called At Least N (ALN) is based on the Effect Predicate Heuristic (EPH) that provides all effect predicate for ISA 5.2 diagrams. The ALN and EPH have been incorporated into the Gungnir tool that was built using model-based testing concepts. The Gungnir uses timed automata to model the specification, in the ISA 5.2 diagrams, and the implementation, in the Ladder language. The timed automata models are automatically extracted, data tests are generated and the tool automatically verifies if the implementation is in conformance with the specification, given a quality level defined by the user.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25
Fig. 26
Fig. 27
Fig. 28
Fig. 29
Fig. 30

Source: [11], p 485

Fig. 31
Fig. 32
Fig. 33
Fig. 34
Fig. 35
Fig. 36
Fig. 37
Fig. 38
Fig. 39
Fig. 40
Fig. 41
Fig. 42
Fig. 43

Similar content being viewed by others

Notes

  1. https://gitlab.com/rodrigopex/gungnir/wikis/home.

  2. X means that the value of the variable in the vector is irrelevant. For example, X0 can be 00 or 10, because there is no change in the output value.

  3. https://gitlab.com/rodrigopex/gungnir/tree/master/resource.

  4. https://gitlab.com/rodrigopex/gungnir/tree/master/src.

  5. https://gitlab.com/rodrigopex/gungnir/tree/master/resource.

  6. https://gitlab.com/rodrigopex/gungnir/wikis.

  7. https://gitlab.com/rodrigopex/gungnir/wikis/Error-detection.

References

  1. AbouTrab, M.S., Brockway, M., Counsell, S., Hierons, R.M.: Testing real-time embedded systems using timed automata based approaches. J. Syst. Softw. 86(5), 1209–1223 (2013). https://doi.org/10.1016/j.jss.2012.12.030

    Article  Google Scholar 

  2. Abrial, J.R.: The B-Book: Assigning Programs to Meanings. Cambridge University Press, New York (1996)

    Book  MATH  Google Scholar 

  3. Alur, R.: Timed Automata. Theor. Comput. Sci. 126, 183–235 (1999)

    Article  MATH  Google Scholar 

  4. Alur, R., Dill, D.L.: A theory of timed automata. Theor. Comput. Sci. 126, 183–235 (1994)

    Article  MathSciNet  MATH  Google Scholar 

  5. Aspar, Z., Shaikh-Husin, N., Khalil-Hani, M.: Algorithm to convert programmable logic controller ladder logic diagram models to petri net models. In: 2015 IEEE Student Conference on Research and Development (SCOReD), pp. 156–161. IEEE (2015)

  6. Bao, J., Wu, H., Yan, Y.: A fault diagnosis system-plc design for system reliability improvement. Int. J. Adv. Manuf. Technol. 75(1), 523–534 (2014). https://doi.org/10.1007/s00170-014-6166-z

    Article  Google Scholar 

  7. Behrmann, G., David, A., Larsen, P.K.: A tutorial on uppaal. In: Bernardo, M., Corradini, F. (eds.) LNCS, Formal Methods for the Design of Real-Time Systems (Revised Lectures), vol. 3185, pp. 200–237. Springer, Berlin (2004a)

    Chapter  Google Scholar 

  8. Behrmann, G., David, A., Larsen, K.G.: A tutorial on uppaal. In: M. Bernardo, F. Corradini (eds.) Formal Methods for the Design of Real-Time Systems: 4th International School on Formal Methods for the Design of Computer, Communication, and Software Systems, SFM-RT 2004, no. 3185 in LNCS, pp. 200–236. Springer (2004)

  9. Bengtsson, J., Yi, W.: Timed automata: semantics, algorithms and tools. Lect. Concurr. Petri Nets 1633, 8–22 (2004)

    MATH  Google Scholar 

  10. Biallas, S., Brauer, J., Kowalewski, S.: Arcade.plc: A verification platform for programmable logic controllers. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, ASE 2012, pp. 338–341. ACM, New York, NY, USA (2012). https://doi.org/10.1145/2351676.2351741

  11. Bryan, L.A., Bryan, E.A.: Programmable Controllers: Theory and Implementation. Industrial Text Company. Illustrator-Kory, Gina (1997)

  12. Chang, J.R., Huang, C.Y.: A study of enhanced mc/dc coverage criterion for software testing. In: 31st Annual International Computer Software and Applications Conference, 2007 (COMPSAC 2007), vol. 1, pp. 457 –464 (2007). https://doi.org/10.1109/COMPSAC.2007.44

  13. Chilenski, J., Miller, S.: Applicability of modified condition/decision coverage to software testing. Softw. Eng. J. 9(5), 193–200 (1994)

    Article  Google Scholar 

  14. de Vasconcelos Oliveira, K., Perkusich, A., Lima, A.M.N., Gorgônio, K., da Silva, L.D.: Standard-based formal validation of programmable logic controller programs. In: 2010 IEEE International Conference on Industrial Technology (ICIT), pp. 1655–1660. IEEE (2010)

  15. de Vasconcelos Oliveira, K., da Silva, L.D., Perkusich, A., Lima, A.M.N., Gorgônio, K.: Automatic timed automata extraction from ladder programs for model-based analysis of control systems. In: 2010 IEEE International Symposium on Industrial Electronics, pp. 90–95. IEEE (2010)

  16. Dreossi, T., Dang, T., Donzé, A., Kapinski, J., Jin, X., Deshmukh, J.V.: Efficient guiding strategies for testing of temporal properties of hybrid systems. In: K. Havelund, G. Holzmann, R. Joshi (eds.) NASA Formal Methods: 7th International Symposium, NFM 2015, Pasadena, CA, USA, April 27–29, 2015, Proceedings, pp. 127–142. Springer International Publishing, Cham (2015)

  17. Goble, W.M., Cheddie, H.: Safety Instrumented Systems Verification: Practical Probabilistic Calculations. ISA (2005)

  18. Hasanain, W., Labiche, Y., Gheorghe, S.: Automated state-based online testing real-time embedded software with rtedge. In: 2015 3rd International Conference on Model-Driven Engineering and Software Development (MODELSWARD), pp. 294–302 (2015)

  19. Hessel, A., Larsen, K., Mikucionis, M., Nielsen, B., Pettersson, P., Skou, A.: Testing Real-Time systems using UPPAAL. In: Formal Methods and Testing, pp. 77–117. Springer (2008)

  20. Hessel, A., Pettersson, P.: Cover—a real-time test case generation tool. In: Proceedings of the 19th IFIP International Conference on Testing of Communicating Systems and 7th International Workshop on Formal Approaches to Testing of Software (2007)

  21. Iqbal, M.Z., Arcuri, A., Briand, L.: Environment modeling and simulation for automated testing of soft real-time embedded software. Softw. Syst. Model. 14(1), 483–524 (2015). https://doi.org/10.1007/s10270-013-0328-6

    Article  Google Scholar 

  22. ISA: Binary Logic Diagrams for Process Operations. ISA—The Instrumentation, Systems, and Automation Society, ISA 5.2-1976 (R1992) edn (1992)

  23. Clarke Jr., E.M., Grumberg, O., Peled, D.A.: Model Checking. The MIT Press, Cambridge (1999)

    MATH  Google Scholar 

  24. Kosmatov, N., Legeard, B., Peureux, F., Utting, M.: Boundary coverage criteria for test generation from formal models. In: 15th International Symposium on Software Reliability Engineering, pp. 139 – 150 (2004). https://doi.org/10.1109/ISSRE.2004.12

  25. Kuzmin, E.V., Sokolov, V.A.: On construction and verification of PLC programs. Autom. Control Comput. Sci. 47(7), 443–451 (2013). https://doi.org/10.3103/S0146411613070110

    Article  Google Scholar 

  26. Kuzmin, E.V., Sokolov, V.A.: Modeling, specification and construction of PLC-programs. Autom. Control Comput. Sci. 48(7), 554–563 (2014). https://doi.org/10.3103/S0146411614070244

    Article  Google Scholar 

  27. Larsen, K.G., Mikucionis, M., Nielsen, B.: Uppaal Tron User Manual (2009). In http://www.cs.aau.dk/~marius/tron/

  28. Larsen, K.G., Mikucionis, M., Nielsen, B., Skou, A.: Testing real-time embedded software using uppaal-tron: an industrial case study. In: EMSOFT ’05: Proceedings of the 5th ACM international conference on Embedded software, pp. 299–306. ACM, New York, NY, USA (2005). http://doi.acm.org/10.1145/1086228.1086283

  29. Lo, H., Peleska, J.: Timed moore automata: Test data generation and model checking. In: 2010 Third International Conference on Software Testing, Verification and Validation, pp. 449 –458 (2010). https://doi.org/10.1109/ICST.2010.60

  30. Ovatman, T., Aral, A., Polat, D., Ünver, A.O.: An overview of model checking practices on verification of plc software. Softw. Syst. Model. 15(4), 937–960 (2016). https://doi.org/10.1007/s10270-014-0448-7

    Article  Google Scholar 

  31. Parr, A.: Programmable Controllers an Engineers Guide, 3rd edn. Newnes, Oxford (2003)

    Google Scholar 

  32. PLCopen: IEC 61131-3: a standard programming resource. PLCopen For Efficiency in Automation (2004). http://www.plcopen.org/

  33. Quezada, J.C., Medina, J., Flores, E., Seck Tuoh, J.C., Solís, A.E., Quezada, V.: Simulation and validation of diagram ladder—petri nets. Int. J. Adv. Manuf. Technol. 1–13 (2016). https://doi.org/10.1007/s00170-016-8638-9

  34. Spivey, J.M.: The Z Notation: A Reference Manual. Prentice Hall, Hertfordshire (1992)

    MATH  Google Scholar 

  35. Tahat, L., Vaysburg, B., Korel, B., Bader, A.: Requirement-based automated black-box test generation. In: 25th Annual International Computer Software and Applications Conference. COMPSAC 2001, pp. 489 –495 (2001). https://doi.org/10.1109/CMPSAC.2001.960658

  36. Utting, M., Legeard, B.: Practical Model-Based Testing: A Tools Approach. Morgan Kaufmann Publishers Inc., San Francisco (2006)

    Google Scholar 

  37. Waez, M.T.B., Dingel, J., Rudie, K.: A survey of timed automata for the development of real-time systems. Computer Science Review 9, 1–26 (2013). https://doi.org/10.1016/j.cosrev.2013.05.001

    Article  MATH  Google Scholar 

  38. Yu, Y., Lau, M.: Comparing several coverage criteria for detecting faults in logical decisions. In: Fourth International Conference on Quality Software, 2004 (QSIC 2004), Proceedings, pp. 14 – 21 (2004). https://doi.org/10.1109/QSIC.2004.1357940

Download references

Author information

Authors and Affiliations

  1. Instituto de Computacão, Universidade Federal de Alagoas, Maceió, Alagoas, Brazil

    Rodrigo José Sarmento Peixoto & Leandro Dias da Silva

  2. Laboratório de Sistemas Embarcados e Computacão Pervasiva, Unidade Acadêmica de Engenharia Elétrica, Universidade Federal de Campina Grande, Campina Grande, Paraíba, Brazil

    Angelo Perkusich

Authors
  1. Rodrigo José Sarmento Peixoto
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Leandro Dias da Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Angelo Perkusich
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Leandro Dias da Silva.

Additional information

Communicated by Dr . M. Papadakis, S. Ali, and G. Perrouin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peixoto, R.J.S., da Silva, L.D. & Perkusich, A. Model-based testing of software for automation systems using heuristics and coverage criterion. Softw Syst Model 18, 797–823 (2019). https://doi.org/10.1007/s10270-018-0690-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10270-018-0690-5

Keywords

Search

Navigation