Automatic test data generation for path testing using GAs

Abstract

Genetic algorithms (GAs) are inspired by Darwin's the survival of the fittest theory. This paper discusses a genetic algorithm that can automatically generate test cases to test a selected path. This algorithm takes a selected path as a target and executes sequences of operators iteratively for test cases to evolve. The evolved test case will lead the program execution to achieve the target path. To determine which test cases should survive to produce the next generation of fitter test cases, a metric named normalized extended Hamming distance (NEHD, which is used to determine whether the final test case is found) is developed. Based on NEHD, a fitness function named SIMILARITY is defined to determine which test cases should survive if the final test case has not been found. Even when there are loops in the target path, SIMILARITY can help the algorithm to lead the execution to flow along the target path.

Introduction

The basic concepts of genetic algorithms (GAs) were developed by Holland [7], [8]. To date, genetic algorithms are well developed and widely used in the area of artificial intelligence and have been successfully applied in many fields of optimization [17], [18], [19], [20], [21].

It is understood that all occurrences of pre-selected features in a tested program should be executed (e.g. all-branch testing requires that each branch in a program should be tested at least once). It is also understood that manually generating a large number of test cases to fulfill the testing criteria is too great an effort. As a result, certain degree of automation is necessary. As Ould [14] suggested, the automation of test case generation is the most important aspect of automatic testing. Automatic test case generation can keep testers from bias when preparing test cases.

Many automatic test case generators have been developed and many classical searching methods have been used to derive test cases [1], [2], [3], [4], [9]. However, these techniques work only for continuous functions (while most program domains are of discontinuous spaces). This is where GAs step in. GAs include a class of adaptive searching techniques which are able to search a discontinous space.

Genetic algorithms have been used to automatically find a program's longest or shortest execution times. In their paper about testing real-time systems using genetic algorithms [16], Wegener et al. investigated the effectiveness of GAs to validate the temporal correctness of real-time systems by establishing the longest and the shortest execution times. The authors declared that an appropriate fitness function for the genetic algorithms is found, and the fitness function is to measure the execution time in processor cycles. Their experiments using GAs on a number of programs have successfully identified new longer and shorter execution times than those that had been found using random testing or systematic testing. However, they did not show the fitness function in their work.

Genetic algorithms have also been used to search program domains for suitable test cases to satisfy the all-branch testing [6], [10], [11]. In their papers about automatic structural testing using genetic algorithms, Jones et al. showed that appropriate fitness functions are derived automatically for each branch predicate. All branches were covered with two orders of magnitude fewer test cases than random testing. The adequacy of the test cases is improved by the genetic algorithm's ability to generate test cases, which are at or close to the branch predicates.

Consider a predicate such as

If A=B then ⋯

The fitness function for this predicate is based on the Hamming distance${}^{\mathrm{\ast }}$ between A and B. The authors declare that the reciprocal of Hamming distance leads to a suitable fitness function.

${}^{\mathrm{\ast }}$The Hamming distance is a count based on the number of different bits in the bit patterns for A and B (i.e. A⊕B). Hamming distance between two bit patterns of equal length is defined as the number of different bits in the bit patterns. For example, the distance between 000 and 111 is three, while the distance between 010 and 011 is one.

The results of the papers above confirm that genetic algorithms are more efficient and effective than random testing in all-branch testing and finding longest execution time. In this paper, we developed a new metric (which is a fitness function) to determine the distance between the exercised path and the target path. The genetic algorithm with the metric is used to generate test cases for testing the target path. Such path-testing algorithm is designed to:

• select a target path from the program under test,

• sample test cases from the whole domain of the program,

• execute the program with the test cases to exercise paths, and calculate the fitness between the target path and each exercised path, and then

• evolve into new generation of fitter test cases,

• repeat from the step 3 to the step 4 until the fittest test case is found.