A Metric-Based Approach to Assess Class Testability

Abstract

In today’s world people have become highly dependent on software systems for almost everything. Hence it necessitates the need for reliable and quality software. Software quality and reliability can only be achieved if the testing is made effective. One way of making testing effective is to improve software testability. Software testability is an external attribute of the software which provides a guideline for testing. Software testability has been defined by numerous researchers from different points of view. An extensive survey of the literature on software testability was done. During the survey, it was found that there were very few researchers who had analyzed the relationship between source code metrics and test metrics. Hence this study aims at performing an empirical study to evaluate some of the source code metrics that have a bearing on the testing effort for object oriented development. Testing effort in turn provides an insight of software testability. This study is performed at the unit level of testing .i.e class level using JUnit testing framework. JUnit framework allows users to create a test class for every java class. Two open source Java projects have been used for this empirical study. In these systems, JUnit test classes have been written for java classes. This study aims to judge the capability of the object oriented metrics to assess class testability. A correlation is found between object oriented metrics and test metrics. To find the correlation the Spearman’s rank-order correlation coefficient is calculated for each object oriented metric of the java classes and three test metrics of the corresponding test classes. The definition used in this study is by ISO[2] which defines testability as: “attributes of software that bear on the effort needed to validate the software product. The attributes of software that we consider are the source code metrics of object oriented software. Object oriented languages have different features like class, method, inheritance, polymorphism, dynamic binding etc. Hence these features also have a bearing on the testing effort and hence on testability. This study focuses on source code factors only and investigates the effect of source code factors on the testing effort. Through testing effort we can quantify testability using two factors given by Brutink[1]. These two factors are test case generation and test case construction factors. Test case generation factor is defined as the factor which influences the number of required test cases. Test case construction factor is defined as the factor which influences the effort needed to construct a test case. For each of the source code metrics we identify which of them is a test case generation and test case construction factor. This in turn, provides an insight into the required testing effort and testability. Hence through source code metrics we can assess software testability. After finding the results, first of all, we observe that some of the source code metrics are highly correlated amongst themselves. Second observation is that, the test metrics are also correlated. There are four size related metrics: LOC, NOA, NOM and WMC. All the four metrics are correlated to all the three test metrics (Except NOA in one of the case study). We find that high LOC, high NOM and high WMC lead to low testability. We can not categorize NOA as leading to high or low testability because of the inconsistency among results of two case studies. High LCOM leads to low testability. So, more cohesive classes have high testability. There is an inverse but weak correlation between DIT and test metrics because in these java projects as the DIT increases the testing decreases in the sense that the core functionality is tested in the parent class and child classes only test their own specific features. Although this correlation will vary depending upon the testing technique used whether the child class tests the features of its parent class or not which is not the norm. In case of NOC we are not able to draw any conclusion because there is no consistent correlation between NOC and test metrics. Although there can be found a correlation among NOC and test metrics if the testing strategy used makes a thorough testing of parent class with the increase in number of children, with an intention that if parent class is having a fault it does not creep into the child class. We foresee the following future work. First, this experiment should be extended to a large number of systems, using different development methodologies like Test driven development, extreme programming and agile software development. Second, this experiment has been conducted at the class level; it should be extended to package level testing. Third, the number of metrics we have considered are very few. This experiment should be extended to a number of other source code metrics which deal with polymorphism, exception handling etc. Fourth, other strong statistical methods should be used to find the correlation between source code metrics and test metrics.