Abstract
Büchi automaton is one of the most commonly used formalisms for specifying and reasoning linear temporal properties. It is instrumental for developing model-checking algorithms for linear temporal logics. Until now Büchi automaton-based specification is primarily used in linear temporal logic model checking. In this paper, we develop test criteria and techniques essential for testing upon specifications in Büchi automata. These criteria measure the semantic relevancy of test cases to a requirement in Büchi automaton. We define the “weak” and “strong” variants of the criteria based on a test suite’s coverage on the transitions of a Büchi automaton. These criteria may be used to measure the quality of existing test cases with respect to requirements in Büchi automaton, and to drive test-case generation. We develop two algorithms that use an off-the-shelf model checker to automate test-case generation for the proposed test criteria. In our extended computational study we deploy two methodologies to measure the effectiveness of our approach. First, we measure the cross coverage of the transition coverage criteria against other existing test criteria. Second, we use a fault-injection technique to measure the sensitivity of our approach. In both cases, our approach shows a better performance compared with existing test criteria, and a good sensitivity in detecting errors systematically injected to a system. Furthermore, the proposed criteria uncover not only the deficiency of a test suite with respect to a linear temporal requirement, but also that of the requirement itself. We propose an algorithm to refine the requirement using the feedback from test-case generation.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
A., B.E.K.: Model-based testing of automotive systems. In: 2008 International Conference on Software Testing, Verification, and Validation, ICST, pp. 485–493 (2008)
Calvagna, A., Gargantini, A.: A logic-based approach to combinatorial testing with constraints. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 66–83. Springer, Heidelberg (2008)
Chilenski, J.J., Miller, S.P.: Applicability of modified condition/decision coverage to software testing. Software Engineering Journal 9(5), 193–200 (1994)
Clarke, E.M., Emerson, E.A.: Design and Synthesis of Synchronization Skeletons using Branching-Time Temporal Logic. In: Kozen, D. (ed.) Logic of Programs 1981. LNCS, vol. 131, pp. 52–71. Springer, Heidelberg (1982)
Committee, S.: Software Considerations in Airborne Systems and Equipment Certification. Tech. rep., Radio Technical Commission for Aeronautics (1992)
Dahl, O.-J., Dijkstra, E.W., Hoare, C.A.R.: Structured Programming. A.P.I.C. Studies in Data Processing, vol. 8. Academic Press (1972)
Fraser, G., Gargantini, A.: An evaluation of model checkers for specification based test case generation. In: ICST ’09: Proceedings of the 2009 International Conference on Software Testing Verification and Validation, IEEE Computer Society, Washington (2009)
Fujiwara, S., von Bochmann, G., Khendek, F., Amalou, M., Ghedamsi, A.: Test selection based on finite state models. IEEE Trans. Softw. Eng. 17(6) (1991)
Gerth, R., Peled, D., Vardi, M.Y., Wolper, P.: Simple on-the-fly automatic verification of linear temporal logic. In: Protocol Specification Testing and Verification. Chapman & Hall (1995)
Giannakopoulou, D., Lerda, F.: From States to Transitions: Improving Translation of LTL Formulae to Büchi Automata. In: Peled, D.A., Vardi, M.Y. (eds.) FORTE 2002. LNCS, vol. 2529, Springer, Heidelberg (2002)
Heimdahl, M.P.E., Rayadurgam, S., Visser, W.: Specification centered testing. In: Proceedings of the Second International Workshop on Automated Program Analysis, Testing and Verification (2001)
Holzmann, G.J.: The Model Checker SPIN. IEEE Transactions on Software Engineering 23(5), 279–295 (1997)
Hong, H.S., Lee, I., Sokolsky, O., Ural, H.: A temporal logic based theory of test coverage and generation. In: Katoen, J.-P., Stevens, P. (eds.) TACAS 2002. LNCS, vol. 2280, pp. 327–341. Springer, Heidelberg (2002)
Jorgensen, P.C.: Software Testing: A Craftsman’s Approach, 1st edn. CRC Press, Inc., Boca Raton (1995)
Joseph, S.: Fault-Injection through Model Checking via Naive Assumptions about State Machine Synchrony Semantics. Master’s thesis, West Virginia University, Morgantown, West Virginia (1998)
Kamel, M., Leue, S.: Formalization and validation of the General Inter-ORB Protocol (GIOP) using PROMELA and SPIN. International Journal on Software Tools for Technology Transfer (STTT) 2(4) (March 2000)
Kupferman, O., Vardi, M.Y.: Vacuity detection in temporal model checking. International Journal on Software Tools for Technology Transfer (STTT) 4(2) (Februrary 2003)
Michel, M.: Complementation is more difficult with automata on infinite words. CNET, Paris (1988)
Rajan, A.: Coverage metrics for requirements-based testing. Ph.D. thesis, University of Minnesota, Minneapolis, MN, USA (2009)
SIMULINK: Dynamic system simulation for matlab, the mathworks (January 1997)
Tan, L.: State Coverage Metrics for Specification-Based Testing with Büchi Automata. In: Gogolla, M., Wolff, B. (eds.) TAP 2011. LNCS, vol. 6706, pp. 171–186. Springer, Heidelberg (2011)
Tan, L., Sokolsky, O., Lee, I.: Specification-based Testing with Linear Temporal Logic. In: IRI’04, pp. 493–498. IEEE society (2004)
Tsay, Y.-K., Chen, Y.-F., Tsai, M.-H., Wu, K.-N., Chan, W.-C.: GOAL: A Graphical Tool for Manipulating Büchi Automata and Temporal Formulae. In: Grumberg, O., Huth, M. (eds.) TACAS 2007. LNCS, vol. 4424, pp. 466–471. Springer, Heidelberg (2007)
Vardi, M.Y.: Automata-theoretic model checking revisited. In: Cook, B., Podelski, A. (eds.) VMCAI 2007. LNCS, vol. 4349, pp. 137–150. Springer, Heidelberg (2007)
Whalen, M.W., Rajan, A., Heimdahl, M.P.E., Miller, S.P.: Coverage metrics for requirements-based testing. In: Proceedings of the 2006 International Symposium on Software Testing and Analysis, ISSTA 2006, ACM, New York (2006)
Young, M., Pezze, M.: Software Testing and Analysis: Process, Principles and Techniques. John Wiley & Sons (2005)
Zeng, B., Tan, L.: Test Criteria for Model-Checking-Assisted Test Case Generation: A Computational Study. In: International Conference on Information Reuse and Integration. IEEE (2012)
Zeng, B., Tan, L.: A unified framework for evaluating test criteria in model-checking-assisted test case generation. Information Systems Frontiers, 1–12 (April 2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Tan, L., Zeng, B. (2015). Testing with Büchi Automata: Transition Coverage Metrics, Performance Analysis, and Property Refinement. In: Bouabana-Tebibel, T., Rubin, S. (eds) Formalisms for Reuse and Systems Integration. FMI 2014. Advances in Intelligent Systems and Computing, vol 346. Springer, Cham. https://doi.org/10.1007/978-3-319-16577-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-16577-6_1
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16576-9
Online ISBN: 978-3-319-16577-6
eBook Packages: EngineeringEngineering (R0)