Abstract
Microsoft is producing interoperability documentation for Windows client-server and server-server protocols. The Winterop team in the Windows organization is responsible for verifying the documentation to ensure its quality. Various test-driven methods are being applied including, when appropriate, a model- based approach. This talk describes core aspects of the quality assurance process and tools that were put in place, and specifically focuses on model-based testing (MBT), using Microsoft’s Spec Explorer technology. Though MBT has been applied successfully to features and products before, this is the first attempt to use it in such a large scale and in the context of a business-critical area within Microsoft, and to the best of the author’s knowledge throughout the whole industry. Empirical results confirm that MBT not only works and that it scales, provided it is accompanied by good tool support and clear methodological guidance, but also that it delivers significant productivity gains when compared to traditional testing in a similar domain and with the same people resources. The method to measure this is based on the effort per end-to-end tested requirement (including not only test design, but also test harnessing and test execution), over a probe of 9,844 requirements in test suites using MBT and 8,728 requirements in non-MBT test suites, with an average effort of 1.39 person days for MBT-tested requirement, and 2.37 person days for traditionally-tested requirement, documenting a productivity gain of 42%. A detailed account of this work is scheduled to appear in the journal for Software Testing and Verification (STVR) under the title Model-based Quality Assurance of Protocol Documentation: Tools and Methodology.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Grieskamp, W. (2010). Microsoft’s Protocol Documentation Program: A Success Story for Model-Based Testing. In: Bottaci, L., Fraser, G. (eds) Testing – Practice and Research Techniques. TAIC PART 2010. Lecture Notes in Computer Science, vol 6303. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-15585-7_3
Download citation
DOI: https://doi.org/10.1007/978-3-642-15585-7_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-15584-0
Online ISBN: 978-3-642-15585-7
eBook Packages: Computer ScienceComputer Science (R0)