Abstract
Constraint-based test data generators rely on SMT or constraint solvers to automatically generate test data (e.g., Pex, Sage, Gatel, PathCrawler, Euclide). However, for some test data generation requests corresponding to particular test objectives, these tools may fail to deliver the expected test data because they focus on efficiency rather than soundness and completeness. We adopt an opposite view in the development of FocalTest, a test data generation tool for Focalize programs. The goal of the tool is to generate an MC/DC-compliant set of test data over the precondition of user-defined program properties. The development of such a correct-by-construction test data generator requires 1) to provide a formally verified translation of Focalize programs and properties into constraint systems; 2) to introduce a formally verified constraint solver able to solve those constraint systems. This paper is concerned with the first step only where we formally demonstrate with Coq the soundness of the translation of an intermediate functional language into a constraint system. This objective requires to formally define the operational semantics of the source language that features the manipulation of concrete data types via pattern-matching and function calls, constructions that are mirrored in the constraint language. Although such a semantics-oriented formalization is only a first step of a larger goal which is to provide a formally verified constraint-based testing tool, we argue that it is an important contribution to the building of more robust software testing tools.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Ayrault, P., Carlier, M., Delahaye, D., Dubois, C., Doligez, D., Habib, L., Hardin, T., Jaume, M., Morisset, C., Pessaux, F., Rioboo, R., Weis, P.: Trusted software within focal. In: C&ESAR 2008, Computer Electronics Security Applications Rendez-vous, pp. 162–179 (2008)
Brucker, A.D., Wolff, B.: Interactive Testing with HOL-TestGen. In: Grieskamp, W., Weise, C. (eds.) FATES 2005. LNCS, vol. 3997, pp. 87–102. Springer, Heidelberg (2006)
Carlier, M.: Test automatique de propriétés dans un atelier de développement de logiciels sûrs. PhD thesis, CEDRIC Laboratory, Paris, France (2009)
Carlier, M., Dubois, C.: Functional Testing in the Focal Environment. In: Beckert, B., Hähnle, R. (eds.) TAP 2008. LNCS, vol. 4966, pp. 84–98. Springer, Heidelberg (2008)
Carlier, M., Dubois, C., Gotlieb, A.: Constraint reasoning in focaltest. In: Int. Conf. on Soft. and Data Tech. (ICSOFT 2010), Athens (July 2010); Also, CNAM Tech. Report CEDRIC-09-1703, 36 pages (2009)
Charreteur, F., Gotlieb, A.: Constraint-based test input generation for java bytecode. In: 21st IEEE Int. Symp. on Softw. Reliability Eng. (ISSRE 2010), San Jose, CA, USA (November 2010)
Denmat, T., Gotlieb, A., Ducasse, M.: Improving constraint-based testing with dynamic linear relaxations. In: 18th IEEE Int. Symp. on Soft. Reliability Eng. (ISSRE 2007), Trollhttan, Sweden (November 2007)
Coq development team. The Coq proof assistant reference manual, Ver. 8.3 (2009)
Fink, G., Bishop, M.: Property-based testing: A new approach to testing for assurance. ACM SIGSOFT Software Engineering Notes 22(4), 74–80 (1997)
Flanagan, C.: Automatic software model checking via constraint logic. Sci. Comput. Program. 50(1-3), 253–270 (2004)
Gómez-Zamalloa, M., Albert, E., Puebla, G.: Test case generation for object-oriented imperative languages in clp. TPLP 10(4-6), 659–674 (2010)
Gotlieb, A.: Euclide: A constraint-based testing platform for critical c programs. In: Int. Conf. on Soft. Testing, Valid. and Verif. (ICST 2009), Denver (April 2009)
Gotlieb, A., Botella, B., Rueher, M.: A CLP Framework for Computing Structural Test Data. In: Palamidessi, C., Moniz Pereira, L., Lloyd, J.W., Dahl, V., Furbach, U., Kerber, M., Lau, K.-K., Sagiv, Y., Stuckey, P.J. (eds.) CL 2000. LNCS (LNAI), vol. 1861, pp. 399–413. Springer, Heidelberg (2000)
Hayhurst, K., Veerhusen, S., Chilenski, J., Rierson, L.K.: A practical tutorial on modified condition/decision coverage, nasa langley. Technical report (2001)
Leroy, X.: Formal verification of a realistic compiler. Communications of the ACM 52(7), 107–115 (2009)
Maranget, L.: Compiling Lazy Pattern Matching. In: Conference on Lisp and Functional Programming. ACM Press (1992)
Marre, B., Arnould, A.: Test sequences generation from lustre descriptions: Gatel. In: Proc. of the 15th IEEE Conference on Automated Software Engineering (ASE 2000). IEEE CS Press (September 2000)
Podelski, A.: Model Checking as Constraint Solving. In: Palsberg, J. (ed.) SAS 2000. LNCS, vol. 1824, pp. 22–37. Springer, Heidelberg (2000)
Wotawa, F., Nica, M.: On the compilation of programs into their equivalent constraint representation. Informatica 32(4), 359–371 (2008)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Carlier, M., Dubois, C., Gotlieb, A. (2012). A First Step in the Design of a Formally Verified Constraint-Based Testing Tool: FocalTest. In: Brucker, A.D., Julliand, J. (eds) Tests and Proofs. TAP 2012. Lecture Notes in Computer Science, vol 7305. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30473-6_5
Download citation
DOI: https://doi.org/10.1007/978-3-642-30473-6_5
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30472-9
Online ISBN: 978-3-642-30473-6
eBook Packages: Computer ScienceComputer Science (R0)