Mehrdad Saadatmand
Detlef Scholle
ABSTRACT
In this paper we introduce a method for runtime verification of the behavior of a system against state machines models in order to identify inconsistencies between the two. This is achieved by tracking states and transitions at runtime and comparing with the expected behavior of the system captured in the form of state machine models. The goal is to increase our confidence that the order of states at runtime matches what is specified by the models. The method also provides for defect localization by identifying that in the transition between which states a deviation from the expected behavior has occurred. The necessity and importance of the method lies in the fact that in model-based development, models are also used to perform analysis. Therefore, if there is any discrepancy between the behavior of the system at runtime and the models, then the result of model-based analyses which are performed may also be invalid and not applicable for the system anymore. For this purpose, in our method we create executable test cases from state machine models to test the runtime behavior of the system.
- M. Utting and B. Legeard, Practical Model-Based Testing: A Tools Approach. San Francisco, CA, USA: Morgan Kaufmann Publishers Inc., 2007. Google Scholar
Digital Library
- J. Zander, I. Schieferdecker, and P. Mosterman, Model-Based Testing for Embedded Systems, ser. Computational Analysis, Synthesis, and Design of Dynamic Systems. Taylor & Francis, 2011. {Online}. Available: http://books.google.se/books?id=fzgzNW_alD0C Google ScholarDigital Library
- M. Saadatmand, A. Cicchetti, and M. Sjödin, "Design of adaptive security mechanisms for real-time embedded systems," in Procs of ESSoS'12. Springer-Verlag, 2012, pp. 121--134. Google ScholarDigital Library
- M. Utting, A. Pretschner, and B. Legeard, "A taxonomy of model-based testing approaches," Journal of Software Testing, Verification and Reliability, vol. 22, no. 5, pp. 297--312, 2012. Google ScholarDigital Library
- J. Peleska, "Industrial-strength model-based testing - state of the art and current challenges," in Proceedings of the Eighth Workshop on Model-Based Testing, March 2013, pp. 3--28.Google Scholar
- MBAT Project: Combined Model-based Analysis and Testing of Embedded Systems, http://www.mbat-artemis.eu/home/, Accessed: November 2013.Google Scholar
- M. Saadatmand and M. Sjödin, "On combining model-based analysis and testing," in Tenth International Conference on Information Technology: New Generations (ITNG), Las Vegas, NV, USA, 2013, pp. 260--266. Google ScholarDigital Library
- S. Anwar, "An anti-lock braking control system for a hybrid electromagnetic/electrohydraulic brake-by-wire system," in American Control Conference, 2004. Proceedings of the 2004, vol. 3, 2004, pp. 2699--2704 vol.3.Google Scholar
- Enea, "The Architectural Advantages of Enea OSE in Telecom Applications," http://www.enea.com/software/solutions/rtos/, Last Accessed: November 2013.Google Scholar
- R. Marinescu, M. Saadatmand, A. Bucaioni, C. Seceleanu, and P. Pettersson, "East-adl tailored testing: From system models to executable test cases," Mälardalen University, Technical Report ISSN 1404-3041 ISRN MDH-MRTC-278/2013-1-SE, August 2013. {Online}. Available: http://www.mrtc.mdh.se/index.php?choice=publications&id=3373Google Scholar
- "EAST-ADL," http://www.east-adl.info/Specification.html, Last Accessed: November 2013.Google Scholar
- R. Alur and D. L. Dill, "A theory of timed automata," Theoretical Computer Science, vol. 126, no. 2, pp. 183--235, 1994. Google ScholarDigital Library
- J. Bengtsson and W. Yi, "Timed automata: Semantics, algorithms and tools," in In Lecture Notes on Concurrency and Petri Nets, ser. Lecture Notes in Computer Science vol 3098, W. Reisig and G. Rozenberg, Eds. Springer--Verlag, 2004.Google Scholar
- J. Håkansson, J. Carlson, A. Monot, P. Pettersson, and D. Slutej, "Component-based design and analysis of embedded systems with uppaal port," in Automated Technology for Verification and Analysis. Springer, 2008, pp. 252--257. Google ScholarDigital Library
- G. Behrmann, R. David, and K. G. Larsen, "A tutorial on Uppaal 4.0," http://www.it.uu.se/research/group/darts/papers/texts/new-tutorial.pdf, November 2006.Google Scholar
- Uppaal for Testing Real-Time Systems Online (TRON), http://people.cs.aau.dk/~marius/tron/, Accessed: November 2013.Google Scholar
- K. G. Larsen, M. Mikucionis, B. Nielsen, and A. Skou, "Testing real-time embedded software using uppaal-tron: an industrial case study," in Proceedings of the 5th ACM international conference on Embedded software, ser. EMSOFT '05. New York, NY, USA: ACM, 2005, pp. 299--306. Google ScholarDigital Library
- M. Saadatmand and M. Sjödin, "Testing of timing properties in real-time systems: Verifying clock constraints," in 20th Asia-Pacific Software Engineering Conference (APSEC), Bangkok, Thailand, December 2013. Google ScholarDigital Library
- W. Eric Wong and Vidroha Debroy, "A survey of software fault localization," Technical Report UTDCS-45-09, The University of Texas at Dallas, November 2009.Google Scholar
- M. A. Alipour, "Automated fault localization techniques; a survey," Technical report, Oregon State University, 2012.Google Scholar
- J. A. Jones and M. J. Harrold, "Empirical evaluation of the tarantula automatic fault-localization technique," in Proceedings of the 20th IEEE/ACM International Conference on Automated Software Engineering, ser. ASE '05. New York, NY, USA: ACM, 2005, pp. 273--282. Google ScholarDigital Library
- C. Liu, X. Yan, L. Fei, J. Han, and S. P. Midkiff, "Sober: Statistical model-based bug localization," SIGSOFT Softw. Eng. Notes, vol. 30, no. 5, pp. 286--295, Sep. 2005. Google ScholarDigital Library
- N. Ge, S. Nakajima, and M. Pantel, "Hidden markov model based automated fault localization for integration testing," in Software Engineering and Service Science (ICSESS), 2013 4th IEEE International Conference on, 2013, pp. 184--187.Google Scholar
- M. Research, "Abstract state machine language (asml)," http://research.microsoft.com/en-us/projects/asml/, Accessed: January 2014.Google Scholar
- H. Pei-Breivold, I. Crnkovic, and M. Larsson, "A systematic review of software architecture evolution research," Journal of Information and Software Technology, July 2011.Google Scholar
- Alten/Xdin AB, http://www.alten.se/en/om-alten/nyheter/the-acquisition-of-enea-experts-is-now-complete/, Accessed: January 2014.Google Scholar
- ITS-EASY post graduate industrial research school for embedded software and systems, http://www.mrtc.mdh.se/projects/itseasy/, Accessed: November 2013.Google Scholar
- VINNOVA: Swedish Governmental Agency for Innovation Systems, http://www.vinnova.se/en/, Accessed: November 2013.Google Scholar
Index Terms
- Runtime verification of state machines and defect localization applying model-based testing
Recommendations
Alternative Effort-optimal Model-based Strategy for State Machine Testing of IoT Systems
WSSE '20: Proceedings of the 2nd World Symposium on Software EngineeringTo effectively test parts of the Internet of Things (IoT) systems having a character of a state machine, Model-based Testing (MBT) approach can be taken. In MBT, a model of a system is created, and test cases generated automatically from the model, and ...
Model-based Runtime Verification Framework
Model-based runtime verification is an extension to the state-of-the-art runtime verification, aimed at checking at runtime the system implementation against the system model (consistency checking) and the system model against the system specification (...
Exhaustive property oriented model-based testing with symbolic finite state machines
AbstractWe advocate a fusion of property-oriented testing (POT) and model-based testing (MBT). The existence of a symbolic finite state machine (SFSM) model fulfilling the properties of interest is exploited for property-directed test data generation and ...
Highlights- We present a test suite generation procedure for model-based property-oriented testing. Properties are specified in LTL.
- Passing a test suite implies that the implementation fulfils every safety property over a given set of atomic ...
Comments