Abstract
Verification of a test case for testing the conformance of protocol implementations against the formal description of the protocol involves verifying three aspects of the test case: expected input/output test behavior, test verdicts, and the test purpose. We model the safety and liveness properties of a test case using branching time temporal logic. There are four types of safety properties: transmission safety, reception safety, synchronization safety, and verdict safety. We model a test purpose as a liveness property and give a set of notations to formally specify a test purpose. All these properties expressed as temporal formulas are verified using model checking on an extended state machine graph representing the composed behavior of a test case and protocol specification. This methodology is shown to be effective in finding errors in manually developed conformance test suites.
Similar content being viewed by others
References
K. Naik and B. Sarikaya Testing communication protocols.IEEE Software, 27–37, 1992
ISO/IEC 9646: Information Technology-Open Systems Interconnection-Conformance Testing Methodology and Framework, 1991.
ISO/IEC IS8807: LOTOS, a formal description technique based on the temporal ordering of observable behavior, ISO/TC97/SC21/WG1-FDT/SC-C, June 1988.
T. Bolognesi and E. Brinksma. Introduction to ISO specification language LOTOS.Computer Networks and ISDN Systems, 25–59 1987.
ISO/IEC IS9074: Estelle — A formal description technique based on an extended state transition model, ISO/TC97/SC21/WG1, 1987.
S. Budkowski and P. Dembinski. An introduction to Estelle: A specification language for distributed systems.Computer Networks and ISDN Systems, 14: 3–23, 1987.
CCITT, Specification and Description Language SDL, Recommendation Z.100, 1992.
ISO/IEC 8824: Profile of abstract syntax notation one, IS8824, 1987.
D.P. Sidhu and T.K. Leung. Formal methods for protocol testing: A detailed study.IEEE Trans. on Software Engineering, 15 (4): 413–426, 1989.
A.T. Dahbura, T.K. Sabnani, and M.U. Uyar. Formal methods for generating protocol conformance test sequencesProceedings of the IEEE, 78 (8): 1317–1326, 1990.
B. Sarikaya, G.v. Bochmann, E. Cerny. A test design methodology for protocol testing.IEEE Trans. on Software Eng., 13(5): 518–526, 1989.
P. Tripathy and B. Sarikaya. Test case generation from LOTOS specification.IEEE Trans. on Computers, 40: 543–552, 1991.
Abstract test suite for transport protocol class 2. The National Computing Centre Limited, Manchester, UK, 1988.
J.B. Goodenough and S.L. Gerhart. Toward a theory of test data selection.IEEE Trans. on Sofware Eng., SE-1 (2): 20–37, 1975.
B. Sarikaya. Conformance testing: Architectures and test sequences.Computer Networks and ISDN Systems, 17: 111–126, 1989.
D. Brand and P. Zafiropulo. On communicating finite-state machines.JACM 30, (2): 323–342, 1983.
K. Naik and B. Sarikaya. An extended finite state machine model for TTCN. Proc. of the 15th. Biennial Symposium on Communications, Kingston, Ontario, 1990, pp. 296–299.
Information Processing Systems-Open System Interconnection-Basic Reference Model, ISO 7498, 1984.
M. Jackson.System development. Prentice Hall, 1983.
K. Naik.Verification of test cases for protocol conformance testing. Ph.D. theseis, Concordia University, Montreal, 1992.
G. v. Bochmann. Hardware specification with temporal logic: An example.IEEE Trans. on Computers, C-31: 223–231, 1982.
L. Lamport. Specifying concurrent program modules.ACM TOPLAS, 5, (2): 190–222, 1983.
E.M. Clarke, E.A. Emerson, and A.P. Sistla. Automatic verification of finite state concurrent systems using temporal logic specifications.ACM TOPLAS, 8 (2), 224–263, 1986.
K. Sabnani, An algorithm technique for protocol verification.IEEE Transaction on Comm, COM-36 (8): 924–931, 1988.
M. Ben-Ari. A. Pnueli, and Z. Manna. The temporal logic of branching time.Acta Informatica, 20: 207–225, 1983.
J.C. Fernandez, J.L. Richier, and J. Voiron. Verification of protocol specifications using the CESAR system. IFIP PSTV V, 1985.
P. Tripathy.A Unified Model for test generation for communication protocols. Ph.D. thesis, Concordia University, Montreal, 1992.
C.H. West. General techniques for communication protocol validation.IBM Journal of Res. and Development, 22 (4): 393–404, 1978.
J. Rubin and C.H. West. An improved protocol validation technique.Computer Networks, 6: 65–73, 1982.
P. Zafiropulo, C.H. West, H. Rudin, D.D. Cowan, and D. Brand. Towards analyzing and synthesizing protocols.IEEE Trans. on Comm., COM-28 (4): 651–661, 1980.
B. Sarikaya and G. v. Bochmann. Synchronization and specification issues in protocol testing.IEEE Trans. on Comm. COM-32 (4): 389–395, 1984.
G. v. Bochmann. Specification of a simplified transport protocol using different formal description techniques.Computer and Networks and ISDN Systems, 18: 335–377, 1990.
M. Dubuc and G. v. Bochmann. Translation from TTCN to LOTOS and verification of Test Cases.FORTE'90, Madrid, 1990.
K. Naik and B. Sarikaya. Verification of protocol conformance test cases using reachability analysis.The Journal of Systems Sofware, 19: 41–57, 1992.
U. Bar and J.M. Schneider. Automated validation of TTCN test suites. IFIP PSTV XII, Orlando, FL, pp. 279–295, 1992.
L. Logrippo, et al. An interpreter for LOTOS, A specification language for distributed systems.Software Practice and Experience, 18: 365–385, 1988.
F.J. Lin, P.M. Chu, and M.T. Liu. Protocol verification using reachability analysis: The state space explosion problem and relief strategies. SIGCOMM'87, Stowe, Vermont, 126–135, 1987.
J.S. Ostroff. Deciding properties of timed transition models.IEEE Trans. on Parallel and Distributed Systems 1 (2): 170–183, 1990.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Naik, K., Sarikaya, B. Test case verification by model checking. Form Method Syst Des 2, 277–321 (1993). https://doi.org/10.1007/BF01384135
Issue Date:
DOI: https://doi.org/10.1007/BF01384135