Skip to main content
SpringerLink
Log in

KeYGenU: combining verification-based and capture and replay techniques for regression unit testing

  • Original Article
  • Published:
International Journal of System Assurance Engineering and Management Aims and scope Submit manuscript

Abstract

Unit testing plays a major role in the software development process. Two essential criteria to achieve effective unit testing are: (1) testing each unit in isolation from other parts of the program and (2) achieving high code coverage. The former requires a lot of extra work such as writing drivers and stubs, whereas the latter is difficult to achieve when manually writing the tests. When changing existing code it is advocated to run the unit tests to avoid regression bugs. However, in many cases legacy software has no unit tests. Writing those tests from scratch is a hard and tedious process, which might not be cost-effective. This paper presents a tool chain approach that combines verification-based testing (VBT) and capture and replay (CaR) test generation methods. We have built a concrete tool chain, KeYGenU, which consists of two existing tools—KeY and GenUTest. The KeY system is a deductive verification and test-generation tool. GenUTest automatically generates JUnit tests for a correctly working software. This combination provides isolated unit test suites with high code-coverage. The generated tests can also be used for regression testing.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16

Similar content being viewed by others

Notes

  1. The numbers in italic are used to denote event intervals which are introduced later in this section. In order to make it easier to follow the example, we use dashed lines to denote return from a call even for the constructor.

  2. A path is feasible if it can be executed. An infeasible path cannot be executed because its path condition is unsatisfiable.

References

  • AspectJ. http://www.eclipse.org/aspectj. Visited May 2011

  • Beckert B, Gladisch C (2007) White-box testing by combining deduction-based specification extraction and black-box testing. In: Gurevich Y, Meyer B (eds) Proceedings, tests and proofs, first international conference, TAP 2007, Zurich, Switzerland, volume 4454 of LNCS. Springer, pp 207–216

  • Beckert B, Hähnle R, Schmitt PH (eds) (2007) Verification of object-oriented software: the KeY approach. LNCS 4334. Springer

  • Burdy L, Requet A, Lanet J-L (2003) Java applet correctness: a developer-oriented approach. In: Araki K, Gnesi S, Mandrioli D (eds) Formal methods, international symposium of formal methods Europe, FME 2003, Pisa, Italy, volume 2805 of LNCS. Springer, pp 422–439

  • Burstall RM (1974) Program proving as hand simulation with a little induction. In: Information processing ’74. Elsevier/North-Holland, pp 308–312

  • Clifton M (2004) Advanced unit test, part V—unit test patterns, January (2004). http://www.codeproject.com/gen/design/autp5.asp. Visited May 2011

  • Cok DR, Kiniry J (2004) ESC/Java2: uniting ESC/Java and JML. In: Barthe G, Burdy L, Huisman M, Lanet J-L, Muntean T (eds) Proceedings, construction and analysis of safe, secure, and interoperable smart devices, international workshop, CASSIS 2004, Marseille, France, volume 3362 of LNCS. Springer, pp 108–128

  • Deng X, Robby, Hatcliff J (2006) Kiasan: a verification and test-case generation framework for Java based on symbolic execution. In: Proceedings, leveraging applications of formal methods, second international symposium, ISoLA 2006, Paphos, Cyprus. IEEE Computer Society, pp 137–137

  • Deng X, Robby, Hatcliff J (2007) Kiasan/KUnit: automatic test case generation and analysis feedback for open object-oriented systems. In: TAICPART-MUTATION ’07: Proceedings of the testing: academic and industrial conference practice and research techniques—MUTATION. Washington, DC, USA. IEEE Computer Society, pp 3–12

  • du Bousquet L, Ledru Y, Maury O, Oriat C, Lanet J-L (2004) Case study in JML-based software validation. In: Proceedings, 19th IEEE international conference on automated software engineering, ASE 2004, Linz, Austria. IEEE Computer Society, pp 294–297

  • EclEmma. http://www.eclemma.org. Visited May 2011

  • Elbaum S, Chin H, Dwyer M, Dokulil J (2006) Carving differential unit test cases from system test cases. In: Young M, Devanbu PT (eds.) Proceedings of the 14th ACM SIGSOFT international symposium on foundations of software engineering, FSE 2006, Portland, OR, USA. ACM, pp 253–264

  • Elbaum SG, Chin HN, Dwyer MB, Jorde M (2009) Carving and replaying differential unit test cases from system test cases. IEEE Trans Softw Eng 35(1):29–45

    Google Scholar 

  • Elssamadisy A (2009) Agile adoption patterns: a roadmap to organizational success. Pearson education

  • Engel C, Hähnle R (2007) Generating unit tests from formal proofs In: Gurevich Y, Meyer B (eds) Proceedings, tests and proofs, first international conference, TAP 2007, Zurich, Switzerland, volume. 4454 of LNCS. Springer, pp 169–188

  • Engel C, Gladisch C, Klebanov V, Rümmer P (2008) Integrating verification and testing of object-oriented software. In: Beckert B, Hähnle R (eds) Proceedings, tests and proofs, second international conference, TAP 2008, Prato, Italy, volume 4966 of LNCS. Springer, pp 182–191

  • Ernst MD (2004) Static and dynamic analysis: synergy and duality. In: Flanagan C, Zeller A (eds.) Proceedings of the 2004 ACM SIGPLAN-SIGSOFT workshop on program analysis for software tools and engineering, PASTE’04. Washington, DC, USA. ACM, p 35

  • Extreme Programming. http://www.extremeprogramming.org. Visited May 2011

  • Fowler M (2000) Refactoring: Improving the Design of Existing Code. Addison-Wesley

  • Gladisch C (2008) Verification-based testing for full feasible branch coverage. In: Cerone A, Gruner S (eds) Proceedings, sixth IEEE international conference on software engineering and formal methods, SEFM 2008, Cape Town, South Africa. IEEE Computer Society, pp 159–168

  • Gladisch C (2011) Verification-based software-fault detection. PhD thesis, Karlsruhe Institute of Technology (KIT). http://digbib.ubka.uni-karlsruhe.de/volltexte/1000023056

  • Gladisch C, Tyszberowicz SS, Beckert B, Yehudai A (2010) Generating regression unit tests using a combination of verification and capture & replay. In: Proceedings, tests and proofs, 4th international conference, TAP2010, Málaga, Spain, volume 6143 of LNCS. Springer, pp 61–76

  • Graves TL, Harrold MJ, Kim J-M, Porter A, Rothermel G (2001) An empirical study of regression test selection techniques. TOSEM 10(2):184–208

    Article  MATH  Google Scholar 

  • Hamill P (2004) Unit test frameworks. O’Reilly

  • Harel D, Kozen D, Tiuryn J (2000) Dynamic logic. MIT Press

  • Harrold MJ, Jones JA, Li T, Liang D, Orso A, Pennings M, Sinha S, Spoon SA, Gujarathi A (2001) Regression test selection for Java software. SIGPLAN Not 36(11):312–326

    Article  Google Scholar 

  • Heisel M, Reif W, Stephan W (1987) Program verification by symbolic execution and induction. In: Morik K (ed) Proceedings, 11th German workshop on artificial intelligence, GWAI 87, volume 152 of Informatik Fachberichte. Springer, pp 201–210

  • Hoare CAR (1969) An axiomatic basis for computer programming. Commun ACM 12(10):576–580

    Article  MATH  Google Scholar 

  • HP WinRunner software. http://www.cbueche.de/WinRunner%20User%20Guide.pdf. Visited May 2011

  • Husted T, Massol V (2003) JUnit in Action. Manning Publications Co.

  • Intel VTune Performance Analyzer http://www.intel.com/cd/software/products/asmo-na/eng/239144.htm. Visited May 2011

  • JUnit. http://www.junit.org. Visited May 2011

  • Laddad R (2003) Aspect J in action: practical aspect-oriented programming. Manning

  • Larsson D, Mostowski W (2004) Specifying Java Card API in OCL. In: Schmitt PH (ed) OCL 2.0 workshop at UML 2003, volume 102 of ENTCS. Elsevier, pp. 3–19, November 2004

  • Mackinnon T, Freeman S, Craig P(2001) Endo-testing: unit testing with mock objects. In: Extreme programming examined. Addison-Wesley, pp 287–301

  • Meyer B (1997) Design by contract: making object-oriented programs that work. In: Proceedings, TOOLS 1997: 25th international conference on technology of object-oriented languages and systems, Melbourne, Australia. IEEE Computer Society, p 360

  • Orso A, Kennedy B (2005) Selective capture and replay of program executions. In: Proceedings of the 2005 workshop on dynamic analysis. pp 1–7

  • Pasternak B, Tyszberowicz S, Yehudai A (2009) GenUTest: a unit test and mock aspect generation tool. J Softw Tools Technol Transfer 11(4):273–290

    Article  Google Scholar 

  • Saff D, Artzi S, Perkins JH, Ernst MD (2005) Automatic test factoring for Java. In: Redmiles DF, Ellman T, Zisman A (eds) Proceedings, 20th IEEE/ACM international conference on automated software engineering, ASE 2005, Long Beach, CA, USA. ACM, pp 114–123

  • Smaragdakis Y, Csallner C (2007) Combining static and dynamic reasoning for bug detection. In: Gurevich Y, Meyer B (eds) Proceedings, tests and proofs, first international conference, TAP 2007, Zurich, Switzerland, volume 4454 of LNCS. Springer, pp 1–16

  • Steven J, Chandra P, Fleck B, Podgurski A (2000) jRapture: a Capture/Replay tool for observation-based testing. In: Proceedings of the international symposium on software testing and analysis, pp 158–167

  • Strichman O (2009) Regression verification: proving the equivalence of similar programs. In: Bouajjani A, Maler O (eds) Proceedings, computer aided verification, 21st international conference, CAV 2009, Grenoble, France, volume 5643 of LNCS. Springer, pp 63–68

  • Taneja K, Xie T (2008) DiffGen: automated regression unit-test generation. In: Proceedings, 23rd IEEE/ACM international conference on automated software engineering, ASE 2008, L’Aquila, Italy. IEEE Computer Society

  • Tillmann N, de Halleux J (2008) Pex-white box test generation for NET. In: Beckert B, Hähnle R (eds) Proceedings, tests and proofs, second international conference, TAP 2008, Prato, Italy, volume 4966 of LNCS. Springer, pp 134–153

  • Tonin I (2007) Verifying the Mondex Case Study—the KeY approach. Technical Report ISSN: 1432-7864, Fakultät für Informatik (Fak. f. Informatik) Institut für Theoretische Informatik (ITI)

  • The Verisoft Project. http://www.verisoft.de. Visited May 2011

  • Visser W, Pasareanu CS, Khurshid S (2004) Test input generation with Java PathFinder. In: Avrunin GS, Rothermel G (eds) Proceedings of the ACM/SIGSOFT international symposium on software testing and analysis, ISSTA 2004, Boston, MA, USA. ACM, pp 97–107

  • Wenzel M, Paulson LC, Nipkow T (2008) The Isabelle framework. In: Mohamed OA, Muñoz C, Tahar S (eds) Proceedings, theorem proving in higher order logics, 21st international conference, TPHOLs 2008, Montreal, Canada, volume 5170 of LNCS. Springer, pp 33–38

  • Xie T (2006) Augmenting automatically generated unit-test suites with regression oracle checking. In: Thomas D (ed) Proceedings, European conference object-oriented programming, ECOOP, Nantes, France, volume 4067 of LNCS. Springer, pp 380–403

  • Yuan H, Xie T (2006) Substra: a framework for automatic generation of integration tests. In: Zhu H, Horgan JR, Cheung S-C, Li JJ (eds) Proceedings of the 2006 international workshop on automation of software test, AST 2006, Shanghai, China. ACM, pp 64–70

Download references

Acknowledgments

We are grateful to Benny Pasternak for modifying GenUTest as needed to combine it with KeY. We also thank Jean-Louis Lanet for providing the banking application that served as our case study.

Author information

Authors and Affiliations

  1. Department of Informatics, Karlsruhe Institute of Technology, Am Fasanengarten 5, 76131, Karlsruhe, Germany

    Bernhard Beckert & Christoph Gladisch

  2. School of Computer Science, The Academic College of Tel Aviv Yaffo, 61083, Tel Aviv Yaffo, Israel

    Shmuel Tyszberowicz

  3. School of Computer Science, Tel Aviv University, 69978, Tel Aviv, Israel

    Amiram Yehudai

Authors
  1. Bernhard Beckert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christoph Gladisch
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shmuel Tyszberowicz
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Amiram Yehudai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shmuel Tyszberowicz.

Additional information

An earlier and shorter version of this paper was presented at the 4th International Conference on Tests and Proofs, Malaga, Spain (Gladisch et al. 2010).

Rights and permissions

Reprints and permissions

About this article

Cite this article

Beckert, B., Gladisch, C., Tyszberowicz, S. et al. KeYGenU: combining verification-based and capture and replay techniques for regression unit testing. Int J Syst Assur Eng Manag 2, 97–113 (2011). https://doi.org/10.1007/s13198-011-0068-3

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13198-011-0068-3

Keywords

Search

Navigation