Margus Veanes
Colin Campbell
ABSTRACT
Online testing is a technique in which test derivation from a model program and test execution are combined into a single algorithm. We describe a practical online testing algorithm that is implemented in the model-based testing tool developed at Microsoft Research called Spec Explorer. Spec Explorer is being used daily by several Microsoft product groups. Model programs in Spec Explorer are written in the high level specification languages AsmL or Spec\#. We view model programs as implicit definitions of interface automata. The conformance relation between a model and an implementation under test is formalized in terms of refinement between interface automata. Testing then amounts to a game between the test tool and the implementation under test.
- Spec Explorer. URL:http://research.microsoft.com/specexplorer, released January 2005.Google Scholar
- R. Alur, T. A. Henzinger, O. Kupferman, and M. Vardi. Alternating refinement relations. In Proceedings of the Ninth International Conference on Concurrency Theory (CONCUR'98), volume 1466 of LNCS, pages 163--178, 1998. Google ScholarDigital Library
- M. Barnett, W. Grieskamp, L. Nachmanson, W. Schulte, N. Tillmann, and M. Veanes. Towards a tool environment for model-based testing with AsmL. In Petrenko and Ulrich, editors, Formal Approaches to Software Testing, FATES 2003, volume 2931 of LNCS, pages 264--280. Springer, 2003.Google Scholar
- M. Barnett, R. Leino, and W. Schulte. The Spec# programming system: An overview. In M. Huisman, editor, Construction and Analysis of Safe, Secure, and Interoperable Smart Devices: International Workshop, CASSIS 2004, volume 3362 of LNCS, pages 49--69. Springer, 2005. Google ScholarDigital Library
- A. Blass, Y. Gurevich, L. Nachmanson, and M. Veanes. Play to test. Technical Report MSR-TR-2005-04, Microsoft Research, January 2005.Google Scholar
- E. Brinksma and J. Tretmans. Testing Transition Systems: An Annotated Bibliography. In Summer School MOVEP'2k -- Modelling and Verification of Parallel Processes, volume 2067 of LNCS, pages 187--193. Springer, 2001. Google ScholarDigital Library
- C. Campbell, W. Grieskamp, L. Nachmanson, W. Schulte, N. Tillmann, and M. Veanes. Model-based testing of object-oriented reactive systems with Spec Explorer. Tech. Rep. MSR-TR-2005-59, Microsoft Research, 2005.Google Scholar
- C. Campbell and M. Veanes. State exploration with multiple state groupings. In D. Beauquier, E. Börger, and A. Slissenko, editors, 12th International Workshop on Abstract State Machines, ASM'05, March 8--11, 2005, Laboratory of Algorithms, Complexity and Logic, Créteil, France, pages 119--130, 2005.Google Scholar
- C. Campbell, M. Veanes, J. Huo, and A. Petrenko. Multiplexing of partially ordered events. In F. Khendek and R. Dssouli, editors, 17th IFIP International Conference on Testing of Communicating Systems, TestCom 2005, volume 3502 of LNCS, pages 97--110. Springer, 2005. Google ScholarDigital Library
- L. de Alfaro. Game models for open systems. In N. Dershowitz, editor, Verification: Theory and Practice: Essays Dedicated to Zohar Manna on the Occasion of His 64th Birthday, volume 2772 of LNCS, pages 269--289. Springer, 2004.Google Scholar
- L. de Alfaro and T. A. Henzinger. Interface automata. In Proceedings of the 8th European Software Engineering Conference and the 9th ACM SIGSOFT Symposium on the Foundations of Software Engineering (ESEC/FSE), pages 109--120. ACM, 2001. Google ScholarDigital Library
- L. Franzen, J. Tretmans, and T. Willemse. Test generation based on symbolic specifications. In J. Grabowski and B. Nielsen, editors, Proceedings of the Workshop on Formal Approaches to Software Testing (FATES 2004), pages 3--17, Linz, Austria, September 2004. To appear in LNCS. Google ScholarDigital Library
- W. Grieskamp, Y. Gurevich, W. Schulte, and M. Veanes. Generating finite state machines from abstract state machines. In ISSTA'02, volume 27 of Software Engineering Notes, pages 112--122. ACM, 2002. Google ScholarDigital Library
- W. Grieskamp, N. Tillmann, and M. Veanes. Instrumenting scenarios in a model-driven development environment. Information and Software Technology, 2004. In press, available online.Google ScholarCross Ref
- Y. Gurevich. Evolving Algebras 1993: Lipari Guide. In E. Börger, editor, Specification and Validation Methods, pages 9--36. Oxford University Press, 1995. Google ScholarDigital Library
- Y. Gurevich, B. Rossman, and W. Schulte. Semantic essence of AsmL. Theoretical Computer Science, 2005. Preliminary version available as Microsoft Research Technical Report MSR-TR-2004-27. Google ScholarDigital Library
- A. Hartman and K. Nagin. Model driven testing - AGEDIS architecture interfaces and tools. In 1st European Conference on Model Driven Software Engineering, pages 1--11, Nuremberg, Germany, December 2003.Google Scholar
- L. Nachmanson, M. Veanes, W. Schulte, N. Tillmann, and W. Grieskamp. Optimal strategies for testing nondeterministic systems. In ISSTA'04, volume 29 of Software Engineering Notes, pages 55--64. ACM, July 2004. Google ScholarDigital Library
- S. Tasiran and S. Qadeer. Runtime refinement checking of concurrent data structures. Electronic Notes in Theoretical Computer Science, 113:163--179, January 2005. Proceedings of the Fourth Workshop on Runtime Verification (RV 2004). Google ScholarDigital Library
- J. Tretmans and A. Belinfante. Automatic testing with formal methods. In EuroSTAR'99: 7th European Int. Conference on Software Testing, Analysis & Review, Barcelona, Spain, November 8--12, 1999.Google Scholar
- J. Tretmans and E. Brinksma. TorX: Automated model based testing. In 1st European Conference on Model Driven Software Engineering, pages 31--43, Nuremberg, Germany, December 2003.Google Scholar
- M. van der Bij, A. Rensink, and J. Tretmans. Compositional testing with ioco. In A. Petrenko and A. Ulrich, editors, Formal Approaches to Software Testing: Third International Workshop, FATES 2003, volume 2931 of LNCS, pages 86--100. Springer, 2004.Google Scholar
Index Terms
- Online testing with model programs
Recommendations
Online testing with model programs
Online testing is a technique in which test derivation from a model program and test execution are combined into a single algorithm. We describe a practical online testing algorithm that is implemented in the model-based testing tool developed at ...
Testing Communication Protocols
The authors describe a unifying, common intermediate model that lets Lotos specifications and TTCN test suites be translated into extended finite-state machines, providing a single medium for conformance testing. The paper covers specifying a protocol ...
Conformance Testing of Formal Semantics Using Grammar-Based Fuzzing
Tests and ProofsAbstractA common problem in verification is to ensure that the formal specification models the real-world system, i.e., the implementation, faithfully. Testing is a technique that can help to bridge the gap between a formal specification and its ...
Comments