Nikolai Tillmann
ABSTRACT
Parameterized unit tests extend the current industry practice of using closed unit tests defined as parameterless methods. Parameterized unit tests separate two concerns: 1) They specify the external behavior of the involved methods for all test arguments. 2) Test cases can be re-obtained as traditional closed unit tests by instantiating the parameterized unit tests. Symbolic execution and constraint solving can be used to automatically choose a minimal set of inputs that exercise a parameterized unit test with respect to possible code paths of the implementation. In addition, parameterized unit tests can be used as symbolic summaries which allows symbolic execution to scale for arbitrary abstraction levels. We have developed a prototype tool which computes test cases from parameterized unit tests. We report on its first use testing parts of the .NET base class library.
- F. Ambert, F. Bouquet, S. Chemin, S. Guenaud, B. Legeard, F. Peureux, N. Vacelet, and M. Utting. BZ-TT: A tool-set for test generation from Z and B using contraint logic programming. In R. Hierons and T. Jerron, editors, FATES 2002 workshop of CONCUR'02, pages 105--120. INRIA Report, August 2002.Google Scholar
- M. Barnett, R. Leino, and W. Schulte. The Spec# programming system: An overview. In M. Huisman, editor, Construction and Analysis of Safe, Secure, and Interoperable Smart Devices: International Workshop, CASSIS 2004, volume 3362 of LNCS, pages 49--69. Springer, 2005. Google ScholarDigital Library
- M. Barnett, D. A. Naumann, W. Schulte, and Q. Sun. 99.44% pure: Useful abstractions in specifications. Conference Proceedings ICIS report NIII-R0426, University of Nijmegen, 2004.Google Scholar
- G. Bernot, M. C. Gaudel, and B. Marre. Software testing based on formal specifications: a theory and a tool. Softw. Eng. J., 6(6):387--405, 1991. Google ScholarDigital Library
- M. Bidoit, H.-J. Kreowski, P. Lescanne, F. Orejas, and D. Sannella, editors. Algebraic system specification and development. Springer-Verlag New York, Inc., New York, NY, USA, 1991. Google ScholarDigital Library
- G. Bierman, M. Parkinson, and A. Pitts. MJ: An imperative core calculus for Java and Java with effects. Technical Report 563, University of Cambridge Computer Laboratory, 2003.Google Scholar
- C. Boyapati, S. Khurshid, and D. Marinov. Korat: automated testing based on Java predicates. In Proc. ISSTA, pages 123--133, 2002. Google ScholarDigital Library
- A. D. Brucker and B. Wolff. Symbolic test case generation for primitive recursive functions. In J. Grabowski and B. Nielsen, editors, FATES, volume 3395 of LNCS, pages 16--32. Springer, 2004. Google ScholarDigital Library
- W. R. Bush, J. D. Pincus, and D. J. Sielaff. A static analyzer for finding dynamic programming errors. Softw. Pract. Exper., 30(7):775--802, 2000. Google ScholarDigital Library
- C. Csallner and Y. Smaragdakis. JCrasher: an automatic robustness tester for Java. Software: Practice and Experience, 34:1025--1050, 2004. Google ScholarDigital Library
- C. Csallner and Y. Smaragdakis. Check 'n' Crash: Combining static checking and testing. In Proc. 27th ICSE, May 2005. Google ScholarDigital Library
- D. Detlefs, G. Nelson, and J. Saxe. Simplify: A theorem prover for program checking. Technical Report HPL-2003-148, HP Labs, Palo Alto, CA, USA, 2003.Google Scholar
- J. Dick and A. Faivre. Automating the generation and sequencing of test cases from model-based specifications. In Proc. FME'93: Industrial Strength Formal Methods, volume 670 of LNCS, pages 268--284. Springer, 1993. Google ScholarDigital Library
- R.-K. Doong and P. G. Frankl. The ASTOOT approach to testing object-oriented programs. ACM Trans. Softw. Eng. Methodol., 3(2):101--130, 1994. Google ScholarDigital Library
- C. Flanagan, K. R. M. Leino, M. Lillibridge, G. Nelson, J. B. Saxe, and R. Stata. Extended static checking for Java. In Proc. 2002 PLDI, pages 234--245. ACM Press, 2002. Google ScholarDigital Library
- P. Godefroid, N. Klarlund, and K. Sen. DART: directed automated random testing. SIGPLAN Notices, 40(6):213--223, 2005. Google ScholarDigital Library
- W. Grieskamp, N. Tillmann, and W. Schulte. XRT --- Exploring Runtime for .NET --- Architecture and Applications. In Proc. 3rd SoftMC, 2005. To appear.Google Scholar
- J. Henkel and A. Diwan. Discovering algebraic specifications from Java classes. In Proc. 17th ECOOP, pages 431--456, 2003.Google ScholarCross Ref
- P. Jalote. Testing the completeness of specifications. IEEE Trans. Softw. Eng., 15(5):526--531, 1989. Google ScholarDigital Library
- R. Jeffries, A. Anderson, and C. Hendrickson. Extreme Programming Installed. Addison Wesley, Boston, MA, USA, Oct. 2000. Google ScholarDigital Library
- J. C. King. Symbolic execution and program testing. Commun. ACM, 19(7):385--394, 1976. Google ScholarDigital Library
- J. Loeckx and K. Sieber. The Foundations of Program Verification, 2nd Edition. Wiley, 1987. Google ScholarDigital Library
- D. Marinov and S. Khurshid. TestEra: A novel framework for automated testing of Java programs. In Proc. 16th ASE, pages 22--31, 2001. Google ScholarDigital Library
- Microsoft. Visual Studio 2005 Team System. http://lab.msdn.microsoft.com/teamsystem/. Google ScholarDigital Library
- J. W. Newkirk and A. A. Vorontsov. Test-Driven Development in Microsoft .NET. Microsoft Press, Apr. 2004. Google ScholarDigital Library
- NUnit development team. NUnit. http://www.nunit.org/.Google Scholar
- Parasoft. Jtest manuals version 5.1. Online manual, July 2004. http://www.parasoft.com/.Google Scholar
- Testing, Verification and Measurement, Microsoft Research. Zap theorem prover. http://research.microsoft.com/tvm/.Google Scholar
- W. Visser, C. S. Pasareanu, and S. Khurshid. Test input generation with Java PathFinder. In Proc. 2004 ISSTA, pages 97--107, 2004. Google ScholarDigital Library
- T. Xie, D. Marinov, W. Schulte, and D. Notkin. Symstra: A framework for generating object-oriented unit tests using symbolic execution. In N. Halbwachs and L. D. Zuck, editors, TACAS, volume 3440 of LNCS, pages 365--381. Springer, 2005. Google ScholarDigital Library
Index Terms
- Parameterized unit tests
Recommendations
Parameterized unit testing: theory and practice
ICSE '10: Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 2Unit testing has been widely recognized as an important and valuable means of improving software reliability, as it exposes bugs early in the software development life cycle. However, manual unit testing is often tedious and insufficient. Testing tools ...
Generating parameterized unit tests
ISSTA '11: Proceedings of the 2011 International Symposium on Software Testing and AnalysisState-of-the art techniques for automated test generation focus on generating executions that cover program behavior. As they do not generate oracles, it is up to the developer to figure out what a test does and how to check the correctness of the ...
Parameterized unit tests
Parameterized unit tests extend the current industry practice of using closed unit tests defined as parameterless methods. Parameterized unit tests separate two concerns: 1) They specify the external behavior of the involved methods for all test ...
Comments