Lev Nachmanson
Margus Veanes
ABSTRACT
This paper deals with testing of nondeterministic software systems. We assume that a model of the nondeterministic system is given by a directed graph with two kind of vertices: states and choice points. Choice points represent the nondeterministic behaviour of the implementation under test (IUT). Edges represent transitions. They have costs and probabilities. Test case generation in this setting amounts to generation of a game strategy. The two players are the testing tool (TT) and the IUT. The game explores the graph. The TT leads the IUT by selecting an edge at the state vertices. At the choice points the control goes to the IUT. A game strategy decides which edge should be taken by the TT in each state. This paper presents three novel algorithms 1) to determine an optimal strategy for the bounded reachability game, where optimality means maximizing the probability to reach any of the given final states from a given start state while at the same time minimizing the costs of traversal; 2) to determine a winning strategy for the bounded reachability game, which guarantees that given final vertices are reached, regardless how the IUT reacts; 3) to determine a fast converging edge covering strategy, which guarantees that the probability to cover all edges quickly converges to 1 if TT follows the strategy.
- R. Alur, C. Courcoubetis, and M. Yannakakis. Distinguishing tests for nondeterministic and probabilistic machines. In Proc. 27th Ann. ACM Symp. Theory of Computing, pages 363--372, 1995.]] Google Scholar
Digital Library
- C. Artho, D. Drusinsky, A. Goldberg, K. Havelund, M. Lowry, C. Pasareanu, G. Rosu, and W. Visser. Experiments with test case generation and runtime analysis. In Börger, Gargantini, and Riccobene, editors, Abstract State Machines 2003, volume 2589 of LNCS, pages 87--107. Springer, 2003.]] Google ScholarDigital Library
- AsmL. URL: http://research.microsoft.com/fse/AsmL/.]]Google Scholar
- M. Barnett, W. Grieskamp, L. Nachmanson, W. Schulte, N. Tillmann, and M. Veanes. Towards a tool environment for model-based testing with AsmL. In Petrenko and Ulrich, editors, Formal Approaches to Software Testing, FATES 2003, volume 2931 of LNCS, pages 264--280. Springer, 2003.]]Google Scholar
- M. Barnett and W. Schulte. Runtime verification of .NET contracts. Elsevier Journal of Systems and Software, 65(3):199--208, 2003.]] Google ScholarDigital Library
- B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, New York, second edition, 1990.]] Google ScholarDigital Library
- D. Box. Code name Indigo: A guide to developing and running connected systems with Indigo. MSDN magazine, 2003. URL: http://msdn.microsoft.com/msdnmag/.]]Google Scholar
- E. Brinksma and J. Tretmans. Testing Transition Systems: An Annotated Bibliography. In Summer School MOVEP'2k -- Modelling and Verification of Parallel Processes, volume 2067 of LNCS, pages 187--193. Springer, 2001.]] Google ScholarDigital Library
- A. Condon. The complexity of stochastic games. Information and Computation, 96(2):203--224, 1992.]] Google ScholarDigital Library
- T. H. Cormen, C. E. Leiserson, and R. L. Rivest. Introduction to Algorithms. MIT Press, 1990.]] Google ScholarDigital Library
- J. Fernandez, C. Jard, T. Jéron, and C. Viho. An experiment in automatic generation of test suites for protocols with verification technology. Science of Computer Programming - Special Issue on COST247, Verification and Validation Methods for Formal Descriptions, 29(1-2):123--146, 1997.]] Google ScholarDigital Library
- J. Filar and K. Vrieze. Competitive Markov decision processes. Springer, Berlin, 1996.]] Google ScholarDigital Library
- A. Gibbons. Algorithmic Graph Theory. Cambridge University Press, 1985.]]Google Scholar
- R. Greenlaw, H. J. Hoover, and W. L. Ruzzo. Limits to Parallel Computation: P-Completeness Theory. Oxford University Press, 1995.]] Google ScholarDigital Library
- W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Generating finite state machines from abstract state machines. In ISSTA'02, volume 27 of Software Engineering Notes, pages 112--122. ACM, 2002.]] Google ScholarDigital Library
- S. Gujiwara and G. V. Bochman. Testing non-deterministic state machines with fault-coverage. In J. Kroon, R. Heijunk, and E. Brinksma, editors, Protocol Test Systems, pages 363--372, 1992.]] Google ScholarDigital Library
- Y. Gurevich. Evolving Algebras 1993: Lipari Guide. In E. Börger, editor, Specification and Validation Methods, pages 9--36. Oxford University Press, 1995.]] Google ScholarDigital Library
- A. Hartman and K. Nagin. Model driven testing - AGEDIS architecture interfaces and tools. In 1st European Conference on Model Driven Software Engineering, pages 1--11, Nuremberg, Germany, December 2003.]]Google Scholar
- N. Immerman. Number of quantifiers is better than number of tape cells. Journal of Computer and System Sciences, 22(3):384--406, 1981.]]Google ScholarCross Ref
- N. D. Jones and W. T. Laaser. Complete problems for deterministic polynomial time. Theoretical Computer Science, 3(1):105--117, 1976.]]Google ScholarCross Ref
- V. V. Kuliamin, A. K. Petrenko, A. S. Kossatchev, and I. B. Bourdonov. UniTesK: Model based testing in industrial practice. In 1st European Conference on Model Driven Software Engineering, pages 55--63, Nuremberg, Germany, December 2003.]]Google Scholar
- D. Lee and M. Yannakakis. Principles and methods of testing finite state machines -- a survey. In Proceedings of the IEEE, volume 84, pages 1090--1123, Berlin, Aug 1996.]]Google ScholarCross Ref
- K. Mehlhorn and G. Schäfer. A heuristic for Dijkstra's algorithm with many targets and its use in weighted matching algorithms. In Meyer, editor, Algorithms - ESA 2001: 9th Annual European Symposium, Aarhus, Denmark, August 28-31, 2001, Proceedings, volume 2161 of LNCS, pages 242--253. Springer, 2001.]] Google ScholarDigital Library
- M. L. Puterman. Markov Decision Processes: Discrete Stochastic Dynamic Programming. Wiley-Interscience, New York, 1994.]] Google ScholarDigital Library
- H. Robinson. Model-based testing home page. URL: http://www.geocities.com/model_based_testing/.]]Google Scholar
- D. P. Sidhu and T.-K. Leung. Formal methods for protocol testing: A detailed study. 15(4):413--426, April 1989.]] Google ScholarDigital Library
- J. Tretmans and A. Belinfante. Automatic testing with formal methods. In EuroSTAR'99: 7th European Int. Conference on Software Testing, Analysis & Review, Barcelona, Spain, November 8--12, 1999.]]Google Scholar
- J. Tretmans and E. Brinksma. TorX: Automated model based testing. In 1st European Conference on Model Driven Software Engineering, pages 31--43, Nuremberg, Germany, December 2003.]]Google Scholar
- W. Yi and K. G. Larsen. Testing probabilistic and nondeterministic processes. In Testing and Verification XII, pages 347--61. North Holland, 1992.]] Google ScholarDigital Library
- F. Zhang and T. Cheung. Optimal transfer trees and distinguishing trees for testing observable nondeterministic finite-state machines. IEEE Transactions on Software Engineering, 29(1):1--14, 2003.]] Google ScholarDigital Library
Index Terms
- Optimal strategies for testing nondeterministic systems
Recommendations
Optimal strategies for testing nondeterministic systems
This paper deals with testing of nondeterministic software systems. We assume that a model of the nondeterministic system is given by a directed graph with two kind of vertices: states and choice points. Choice points represent the nondeterministic ...
Adaptive Testing of Nondeterministic Systems with FSM
HASE '14: Proceedings of the 2014 IEEE 15th International Symposium on High-Assurance Systems EngineeringThe paper addresses the problem of testing systems considered as black boxes and modelled by Finite State Machines (FSM). While the theory of testing from deterministic FSMs has received a lot of attention, testing from nondeterministic machines is not ...
Comments