Skip to main content
Log in

Formalization of test experiments

  • Published:
Programming and Computer Software Aims and scope Submit manuscript

Abstract

Formal methods for testing conformance of the system under examination to its specification are examined. The operational interaction semantics is specified by a special testing machine that formally determines the testing capabilities. A set of theoretically powerful and practically important capabilities is distinguished that can be reduced to the observation of external actions and refusals (the absence of external actions). The novelties are as follows. (1) Parameterization of the semantics by the families of observable and not observable refusals, which makes it possible to take into account various constraints on the (correct) interactions. (2) Destruction as a forbidden action, which is possible but should not be performed in the case of a correct interaction. (3) Modeling of the divergence by the Δ-action, which also should be avoided in the case of a correct interaction. On the basis of this semantics, the concept of safe testing, the implementation safety hypothesis, and the safe conformance relation are proposed. The safe conformance relation corresponds to the principle of independent observations: a behavior of an implementation is correct or incorrect independently of its other possible behaviors. For a more narrow class of interactions, another version of the semantics based on the ready traces may be used along with the corresponding conformance relation. Some propositions concerning the relationships between the conformance relations under various semantics are formulated. The completion transformation that solves the problem of the conformance relation reflexivity and a monotone transformation that solves the monotonicity problem (preservation of the conformance under composition) are defined.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Bernot, G., Testing against Formal Specifications: A Theoretical View, TAPSOFT’91, Abramsky, S. and Maibaum, T.S.F., Eds., vol. 2, pp. 99–119, Lect. Notes Comput. Sci., 1991, vol. 494.

  2. Milner, R., Modal Characterization of Observable Machine Behavior, Proc. CAAP, 1981, Astesiano, G. and Bohm, C. Eds., Lect. Notes Comput. Sci., 1981, vol. 112, pp. 25–34.

  3. Van Glabbeek, R.J., The Linear Time—Branching Time Spectrum, Proc. of CONCUR’90, Baeten, J.C.M. and Klop, J.W., Eds., Lect. Notes Comput. Sci., 1990, vol. 458, pp. 278–297.

  4. Van Glabbeek, R.J., The Linear Time—Branching Time Spectrum II: The Semantics of Sequential Processes with Silent Moves, Proc. of CONCUR’93, Hildesheim, Germany, 1993, Best, E., Ed., Lect. Notes Comput. Sci., 1993, vol. 715, p. 66.

    Google Scholar 

  5. Lynch, N.A. and Tuttle, M.R., Hierarchical Correctness Proofs for Distributed Algorithms, Proc. of the 6th ACM SIGACT-SIGOPS Symp. on Principles of Distributed Computing, 1987, pp. 137–151.

  6. Phalippou, M. Relations d’Implantation et Hypotheses de Test des Automates a Entrees et Sorties, PhD Thesis, l’Universite de Bordeaux 1, 1994.

  7. Tretmans, J., Test Generation with Inputs, Outputs and Repetitive Quiescence, Software Concepts and Tools, 1996, vol. 17, issue 3.

  8. Milner, R., A Calculus of Communicating Systems, Lect. Notes Comput. Sci., 1980, vol. 92.

  9. Milner, R., Communication and Concurrency, Prentice-Hall, 1989.

  10. Vaandrager, F., On the Relationship between Process Algebra and Input/Output Automata, Logic in Computer Science, Sixth Annual IEEE Symposium, IEEE Computer Society, 1991, pp. 387–398.

  11. Jard, C., Jéron, T., Tanguy, L., and Viho, C., Remote Testing Can Be as Powerful as Local Testing, Formal Methods for Protocol Engineering and Distributed Systems, FORTE XII/PSTV XIX’99 Wu, J., Chanson, S., and Gao, Q., Eds., Beijing, 1999, pp. 25–40.

  12. Van der Bijl, M., Rensink, A., and Tretmans, J., Compositional Testing with ioco, Formal Approaches to Software Testing: Third Int. Workshop FATES, Petrenko, A. and Ulrich, A., Eds., Montreal, 2003, Lect. Notes Comput. Sci., 2003, vol. 2931, pp. 86–100.

  13. Van der Bijl, M., Rensink, A., and Tretmans, J., Component Based Testing with ioco, CTIT Technical Report, Univ. of Twente, 2003, no. TR-CTIT-03-34.

  14. Von Bochmann, G.V. and Petrenko, A., Protocol Testing: Review of Methods and Relevance for Software Testing, Proc. of ISSTA, 1994, pp. 109–124.

  15. Lee, D. and Yannakakis, M., Principles and Methods of Testing Finite State Machines: A Survey, Proceedings of the IEEE, vol. 84, no. 8, pp. 1090–1123, Berlin: IEEE Computer Society, 1996.

    Google Scholar 

  16. Tretmans, J., Formal Approaches to Conformance Testing, Ph.D. Thesis, Enschede, Netherlands: Univ. of Twente, 1992.

    Google Scholar 

  17. Bourdonov, I.B. and Kossatchev, A.S., Testing Components of a Distributed System, Trudy Vserossiiskoi konferentsii “Nauchnyi servis v seti Internet” (Proc. of the All-Russia Conf. on the Research Services on the Internet), Moscow: Mosk. Gos. Univ., 2005, pp. 63–65.

    Google Scholar 

  18. Bourdonov, I.B. and Kossatchev, A.S., Verification of the Composition of a Distributed System, Trudy Vserossiiskoi konferentsii “Nauchnyi servis v seti Internet” (Proc. of the All-Russia Conf. on the Research Services on the Internet), Moscow: Mosk. Gos. Univ., 2005, pp. 67–69.

    Google Scholar 

  19. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Formal Conformance Testing of Systems with Refused Inputs and Forbidden Actions, Proc. of MBT, Vienna, 2006.

  20. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Teoriya sootvetstviya dlya sistem s blokirovkami i razrusheniyami (Conformance Theory for Systems with Refusals and Destruction), Moscow: Nauka (in press).

  21. Heerink, L. and Tretmans, J., Refusal Testing for Classes of Transition Systems with Inputs and Outputs, in Formal Description Techniques and Protocol Specification, Testing and Verification, Chapman & Hill, 1997.

  22. Heerink, L. Ins and Outs in Refusal Testing, PhD Thesis, Enschede, Netherlands: Univ. of Twente, 1998.

    Google Scholar 

  23. Petrenko, A., Yevtushenko, N., and Huo, J.L., Testing Transition Systems with Input and Output Testers, Proc. 15th Int. Conf. on Communicating Systems, TestCom’2003, Sophia, Antipolis, France, pp. 129–145.

  24. Lestiennes, G. and Gaudel, M.-C., Test de Systemes Reactifs non Receptifs, J. Europ. des Systemes Automatises, Modelisation des Systemes Reactifs, 2005, pp. 255–270.

  25. Hoare, C.A.R., An Axiomatic Basis for Computer Programming, Commun. ACM, 1969, vol. 12, no. 10, pp. 576–585.

    Article  MATH  Google Scholar 

  26. Blass, A., Gurevich, Y., Nachmanson, L., and Veanes, M., Play to Test Microsoft Research, Technical Report, 2005, no. MSR-TR-2005-04; 5th Int. Workshop on Formal Approaches to Testing of Software (FATES 2005), Edinburgh, 2005.

  27. Fujiwara, S. and von Bochmann, G. Testing Nondeterministic Finite State Machine with Fault Coverage, Proc. IFIP TC6 Fourth Int. Workshop on Protocol Test Systems, 1991, Kroon, J., Heijink, R.J., and Brinksma E., Eds., North-Holland, 1992, pp. 267–280.

  28. Petrenko, A., Yevtushenko, N., and von Bochmann, G., Testing Deterministic Implementations from Nondeterministic FSM Specifications, Selected Proc. of the IFIP TC6 9th Int. Workshop on Testing of Communicating Systems, 1996.

  29. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Application of Finite Automatons for Program Testing, Programmirovanie, 2000, no. 2, pp. 12–28 [Programming Comput. Software (Engl. Transl.), 2000, vol. 26, no. 2, pp. 61–73].

  30. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Irredundant Algorithms for Traversing Directed Graphs: The Deterministic Case, Programmirovanie, 2003, no. 5, pp. 11–30 [Programming Comput. Software (Engl. Transl.), 2003, vol. 29, no. 5, pp. 245–258].

  31. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Irredundant Algorithms for Traversing Directed Graphs: The Nondeterministic Case, Programmirovanie, 2004, no. 1, pp. 4–24 [Programming Comput. Software (Engl. Transl.), 2004, vol. 30, no. 1, pp. 2–17].

  32. Kuliamin, V.V., Petrenko, A.K., Kossatchev, A.S., and Bourdonov, I.B., The UniTesK Approach to Designing Test Suites, Programmirovanie, 2003, no. 6, pp. 25–43 [Programming Comput. Software (Engl. Transl.), 2003, vol. 29, no. 6, pp. 310–322].

  33. Kuliamin, V.V., Kossatchev, A.S., Petrenko, A.K., Pakoulin, N.V., and Bourdonov, I.B., Integration of Functional and Timed Testing of Real-Time and Concurrent Systems, in Perspectives of System Informatics, Lect. Notes Comput. Sci., 2003.

  34. Bourdonov, I.B., Kossatchev, A.S., and Kuliamin, V.V., Asynchronous Automata: Classification and Testing, Tr. ISP RAN (Proceedings of Institute for System Programming, Russian Academy of Sciences), 2003, vol. 4, pp. 7–84.

    Google Scholar 

  35. De Nicola, R. and Hennessy, M.C.B., Testing Equivalence for Processes, Theor. Comput. Sci., 1984, vol. 34, pp. 83–133.

    Article  MATH  Google Scholar 

  36. Langerak, R. A., Testing Theory for LOTOS Using Deadlock Detection, in Protocol Specification, Testing, and Verification IX, Brinksma E., Scollo, G., and Vissers, C.A., Eds. North-Holland, 1990, pp. 87–98.

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to I. B. Bourdonov.

Additional information

Original Russian Text © I.B. Bourdonov, A.S. Kossatchev, V.V. Kuliamin, 2007, published in Programmirovanie, 2007, Vol. 33, No. 5.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Bourdonov, I.B., Kossatchev, A.S. & Kuliamin, V.V. Formalization of test experiments. Program Comput Soft 33, 239–260 (2007). https://doi.org/10.1134/S0361768807050015

Download citation

  • Received:

  • Issue Date:

  • DOI: https://doi.org/10.1134/S0361768807050015

Keywords

Navigation