Static/Dynamic Test Case Generation For Software Upgrades via ARC-B and Deltatest

Abstract

This chapter presents test generation techniques that address the automatic production of test cases to validate evolving software, aiming to improve the adequacy of testing in the light of a performed upgrade. For human experts it is usually hard to achieve high test case coverage by manually generating test cases. In particular, when a program is upgraded, testers need to adapt the test suite of the base version of the program to the new version, to cover the relevant code according to the kind of upgrade that has been implemented. The test case generation techniques presented in this chapter aim to automatically augment the existing test suites with test cases that exercise the uncovered regions of the code. These test cases represent extremely useful executions to give as complete a view as possible of the behavior of the upgraded program. We will describe ARC-B, a technique for the automatic generation of test cases, and its extension as DeltaTest that we have developed in the context of the European FP7 Project PINCETTE. DeltaTest extends ARC-B to target software changes in a more specific fashion, according to ideas that resulted from the feedback gained while using ARC-B during the project. Specifically, DeltaTest exploits a program slicer to distinguish the code impacted by modifications, and builds on this information to generate test suites that specifically address the testing of software changes. In the next sections, we describe the technology that underlies ARC-B, report our experience of applying ARC-B to industrial software provided as case studies by industrial partners of the project, present the DeltaTest technique, and discuss initial data on the strength of DeltaTest.