Christopher Robinson-Mallett
Abstract
This paper considers the problem of testing the communication between components of a timed distributed software system. We assume that communication is specified using timed interface automata and use computational tree logic (CTL) to define coverage criteria that refer to send- and receive-statements and communication paths. Given such a state-based specification of a distributed system and a concrete coverage goal, a model checker is used in order to determine the coverage provided by a finite set of test-cases, expressed using sequence diagrams. If parts of the specification remain uncovered then a goal is derived so that the model checker can be used to generate test cases that increase the coverage provided by the test suite. A major benefit of the presented approach is the generation of a potentially minimal set of test cases with the confidence that every interaction between components is executed during testing. A potential additional benefit of this approach is that it provides a visual description of the state based testing of distributed systems, which may be beneficial in other contexts such as education and program comprehension. The complexity of our approach strongly depends on the input model, the testing goal, and the model checking algorithm, which is implemented in the used tool. While a particular model checker, UPPAAL, was used, it should be relatively straightforward to adapt the approach for use with other CTL based model checkers.
- Alur R.; Dill L., A Theory of Timed Automata. Theoretical Computer Science, 126(2) pp. 183--235. 1994. Google Scholar
Digital Library
- Aizenbud-Reshef N., Coverage Analysis for Message Flows, 12th International Symposium on Software Reliability Engineering (ISSRE 2001), pp. 276--286. IEEE 2001, Google ScholarDigital Library
- Alfaro L. de, Henzinger T. A., Interface Automata. Proc. of the 8th European Software Engineering Conference (ESEC 2001). pp. 109--120. ACM. 2001. Google ScholarDigital Library
- Alfaro L. de, Henzinger T. A., Stroelinga M., Timed Interfaces. Proc. of the Second International Conference on Embedded Software, (EMSOFT 2002), 2491 pp. 108--122. LNCS. Springer, 2002. Google ScholarDigital Library
- Bengtsson J., Larsen K. G., Larsson F., Pettersson P., Yi W., UPPAAL - A Tool Suite for Automatic Verification of Real-Time Systems. Workshop on Verification and Control of Hybrid Systems, DIMACS, 1995. Google ScholarDigital Library
- Bochmann G. v., Petrenko A., Protocol Testing: Review of Methods and Relevance for Software Testing. Proc. of the International Symposium on Software Testing and Analysis (ISSTA 1994), pp. 109--124. ACM Press. 1994. Google ScholarDigital Library
- Cardell-Oliver R., Conformance test experiments for distributed real-time systems. Proc. of the 2002 ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), pp. 159--163. ACM 2002. Google ScholarDigital Library
- Chow T. S., Testing Software Design Modeled by Finite-State Machines. IEEE Transactions on Software Engineering, 4(3) pp. 178--187. 1978.Google ScholarDigital Library
- Clarke E. M., Grumberg O., Peled D. A., Model Checking. MIT Press. Boston. 2000.Google Scholar
- Clarke E. M., Emerson E. A., Design and Synthesis of Synchronization Skeletons using Branching Time Temporal Logic. Proc. of the Workshop on Logic of Programs. Yorktown Heights, NY, LNCS 131 pp. 52--71., Springer Press. 1981. Google ScholarDigital Library
- Gill A., Introduction to the Theory of Finite-State Machines. McGraw Hill. Berkeley. 1962.Google Scholar
- Harrold M. J., Rothermel G., Performing dataflow testing on classes, Proc. Symposium Foundations of Software Engineering, ACM, 1994. Google ScholarDigital Library
- Hennie F. C., Fault-Detecting Experiments for Sequential Circuits. Proc. of the Symposium on Switching Circuit Theory and Logical Design NJ, pages 95--110. 1964.Google Scholar
- Heimdahl M., George D., Weber R., Specification Test Coverage Adequacy Criteria = Specification Test Generation Inadequacy Criteria? Proc. 8th Intern. Symp. High Assurance Systems Engineering (HASE'04), Tampa, F1, IEEE, 2004. Google ScholarDigital Library
- Hong H. S., Cha S. D., Lee I., Sokolsky O., Ural H., Data Flow Testing as Model Checking, Proc. of International Conference on Software Engineering (ICSE '03), pp. 232--242, May 2003. Google ScholarDigital Library
- Hong H., Lee I., Sokolsky O., Ural H., A Temporal Logic Based Theory of Test Coverage and Generation, International Conference on Tools and Algorithms for Construction and Analysis of Systems (TACAS2002), April 8--11, 2002. Google ScholarDigital Library
- Huhn M., Mücke T., Generation of Optimized Test suites for UML Statecharts with Time. Testing of Communicating Systems (TestCom'04). Oxford. Springer. 2004.Google Scholar
- Hutchins M., Foster H., Goradia T., Ostrand T. J., Experiments of the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria. Proc. of Intern. Conf. on Software Engineering, Sorento, Italy, ACM, 1994. Google ScholarDigital Library
- Khoumsi A., A Temporal Approach for Testing Distributed Systems. IEEE Trans. on Software Engineering, 28(11) pp 1085--1103, Nov 2002. Google ScholarDigital Library
- Lee D., Yannakakis M., Testing Finite-State Machines: State Identification and Verification. IEEE Transactions on Computers, 43(3): 306--320. 1994. Google ScholarDigital Library
- Liu W., Dasiewicz P., Component Interaction Testing Using Model Checking, Canadian Conference on Electrical and Computer Engineering, 1 pp. 41--46, IEEE, 2001.Google Scholar
- Moore E. F., Gedanken-Experiments on Sequential Machines. Automata Studies (Annals of Mathematics Studies), 34. 1956.Google Scholar
- Offutt A. J., Xiong Y., Liu S., Criteria for Generating Specification-based Tests, Proc. 5th Intern. Conf. on Engineering of Complex Computer Systems, ACM, 1999 Google ScholarDigital Library
- Prowell S. J., Poore J. H., Computing system reliability using Markov chain usage models, Journal of Systems and Software, 73(2) pp. 219--225, Elsevier, 2004. Google ScholarDigital Library
- Queille J. P., Sifakis J., Specification and Verification of Concurrent Systems in CESAR, Proc. 5th Intern. Symp. on Programming, pp. 337--351. 1982. Google ScholarDigital Library
- Robinson-Mallett C., Mücke T., Liggesmeyer P., Goltz U., Generating Optimal Distinguishing Sequences with a Model Checker. Advances in Model-Oriented Software Testing (A-MOST'05). St. Louis. 2005. Google ScholarDigital Library
- Robinson-Mallett C., Liggesmeyer P., State Identification and Verification using a Model Checker. Proceedings of the Software Engineering 2006. GI Edition Lecture Notes in Informatics, Leipzig, March 2006Google Scholar
- Robinson-Mallett C., Mücke T., Liggesmeyer P., Goltz U., Extended State Identification and Verification using a Model Checker. Journal on Information and Software Technology, 48 (10) pp. 981--992. Elsevier. October 2006Google ScholarCross Ref
Index Terms
- Achieving communication coverage in testing
Recommendations
Using communication coverage criteria and partial model generation to assist software integration testing
This paper considers the problem of integration testing the components of a timed distributed software system. We assume that communication between the components is specified using timed interface automata and use computational tree logic (CTL) to ...
Coverage Based Testing with Test Purposes
QSIC '08: Proceedings of the 2008 The Eighth International Conference on Quality SoftwareTest purposes have been presented as a solution to avoid the state space explosion when selecting test cases from formal models. Although such techniques work very well with regard to the speed of the test derivation, they leave the tester with one ...
Using coverage to automate and improve test purpose based testing
Test purposes have been presented as a solution to avoid the state space explosion when selecting test cases from formal models. Although such techniques work very well with regard to the speed of the test derivation, they leave the tester with one ...
Comments