Viewpoint representation validation: a case study on two metrics from the Chidamber and Kemerer suite

Abstract

A metric definition follows from a comprehensive sequence of steps that takes into account all the requirements of the representational theory of measurement. The last step emphasises that numerical relations must preserve and are preserved by empirical relations or viewpoints. It is the preservation of viewpoints in numerical relations that we have tried to empirically validate for the metrics depth of inheritance tree (DIT) and number of children (NOC), proposed by Chidamber and Kemerer [S.R. Chidamber, C.F. Kemerer, IEEE Trans. Software Eng. 20 (6) (1994) 476–493], and which we also refer to as viewpoint representation validation. An analysis of the results indicates the possibility of the two metrics not preserving the viewpoints adequately.

Introduction

Software measurement has now become an important component of software engineering practice. However, its application can be successful only if the metrics used are theoretically and practically valid (Fenton and Pfleeger, 1996). A metric is theoretically valid if it measures the right attribute and possesses desirable formal (or axiomatic) properties. For instance, according to measurement theory (Roberts, 1979, Fenton and Pfleeger, 1996), metrics must obey the representation condition, which specifies that all intuitively derivable empirical relations must be preserved in the numerical, or representation, domain. Besides the criteria that determine theoretical validity, there are a number of criteria that determine a metric's practical validity, or usefulness. Some of these are (Kearny et al., 1986, Daskalantokanis, 1992): responsiveness to modifications (robustness), availability of a datum (normativeness), influence of attributes other than that measured (specificity), ability to prescribe methods for improvement (prescriptiveness), and understandability. It can be noticed that given the multitude of criteria that must be satisfied, the evaluation of “validity of a metric” is not simple. It requires an in-depth theoretical analysis and empirical investigation (Kitchenham et al., 1995).

In this paper we have described an empirical validation of the object-oriented metrics of depth of inheritance tree (DIT) and number of children (NOC), proposed by Chidamber and Kemerer (1994) (referred to as C&K hereafter), from the point of view of measurement theory. In the validation effort, which centres on the representation condition, we first determine empirical relations on a given set of classes within an inheritance hierarchy, through a viewpoint specification, and then check whether the numerical relations prescribed by the metrics preserve the empirical relations. In the following sections we elaborate on the exact object of our validation and its relation to the representational theory of measurement.

The fundamental concept in object-orientation is that of an object. An object encapsulates a part of its state in data members, descriptions of manipulations in member functions, and exhibits a well-defined behaviour (Korson and McGregor, 1990). Following (Chidamber and Kemerer, 1994), the property set p(x) of an object x is defined as p(x)={M_x}∪{I_x} where {M_x} is the set of member functions and I_x is the set of data members of object x. A member of the property set p(x) of object x is also called a property of x. A class, as a description of one kind of an object, provides a specification and implementation of the properties of its object instances. A distinction is usually made between the specification of the class and its implementation (Booch, 1994). The implementation comprises the internal view and is hidden from other classes, i.e., there is a delineation between specification and implementation to enforce information hiding. This information hiding, though, may be violated through class inheritance.

Class inheritance is fundamentally an object-oriented programming concept (Korson and McGregor, 1990). It is a relation among classes that allows the definition and implementation of one class to be based on one (single inheritance) or more (multiple inheritance) classes. If class X inherits from class Y, X is called the subclass and Y the superclass. In this case X has two parts, a derived part and an incremental part (Korson and McGregor, 1990). The derived part is the part comprising of properties inherited from Y and the incremental part comprises of properties specific to X. The subclass and superclass relations so established are transitive relations.

The reason behind using inheritance is usually the requirement of representing a relation that exists among the key abstractions in the application-domain, and this relation is IS-A (Booch, 1994). Alternatively, the reason behind using inheritance might be no other than code reuse (Armstrong and Mitchell, 1994). Class Y may inherit from a class X to reuse some code in X, so that Y can be developed cheaply and quickly. Whatever be the motivating reason, design choices on an inheritance hierarchy reflect a choice on restricting or expanding the scope of properties of classes of objects in the hierarchy (Chidamber and Kemerer, 1994).

According to C&K (Chidamber and Kemerer, 1994), the ontological notion of “scope of properties” is an attribute of a class that relates it to its existence in an inheritance hierarchy. This attribute can be described in the context of the inheritance hierarchy as capturing the notion of: “how far does the influence of a property extend?” However, this notion can be further extended to two different points of view. Namely, “the extent to which a class is influenced by the properties of its ancestors,” and “the potential impact on descendants.”

From the standpoint of metric development what one would be interested in is, defining metrics that can numerically represent the attribute of “scope of properties” as a combination of these two points of view. C & K have accordingly defined the metrics of DIT and NOC:

• DIT metric of a class is the depth of inheritance of the class. In the case of multiple inheritance, the DIT is the maximum length from the node to the root of the tree.

• NOC metric of a class is the number of immediate subclasses subordinate to the class in the class inheritance hierarchy.

The theoretical basis given for the above two metrics are:

• DIT: It is a measure of how many ancestor classes can potentially affect this class.

• NOC: It is a measure of how many subclasses are going to inherit the members of the parent class.

In order to establish the validity of the metrics, C&K draw on the representational theory of measurement. More specifically, they list a number of viewpoints that were “gathered from an extensive collaboration with a highly experienced team of software engineers from a software development organisation”, and which, according to them, are represented accurately in numerical relations. Whether this is really the case is what we validate in this research. Although, we have not used a team of highly experienced software engineers in our validation effort but, postgraduate students, our work, since it is systematised, we feel, has its importance in trying to establish whether the viewpoints on the two metrics can be treated as conventional wisdom. Also, the methodology described in this paper is general enough to be used for similar validations of other metrics.

Measurement is not just about assigning numbers, but also ensuring that the numbers preserve observed relations. It involves the notion of representation, i.e., translations of `qualitative' concepts into appropriate `quantitative' concepts. More formally in measurement we seek a mapping from an observed or empirical relational system to a formal (numerical) relational system that `preserves' all the relations and operations (Roberts, 1979, Briand et al., 1995, Fenton and Pfleeger, 1996).

A relational system is defined as $\text{A}\text{=(A,R}{}_1\text{,…R}{}_{\mathrm{n}}\text{,o}{}_1\text{,…,o}{}_{\mathrm{m}}\text{)}$, where A is an non-empty set of entities, R_i,i=1,…,n, are relations in A, and $\text{o}{}_{\mathrm{j}}\text{,}\text{j=1,…,m}$, are closed binary operations in A (Briand et al., 1995). An empirical relational system is formally specified as $\text{E}\text{=(E,R}{}_1\text{,…R}{}_{\mathrm{n}}\text{,o}{}_1\text{,…,o}{}_{\mathrm{m}}\text{)}$, where E is the non-empty set of entities whose attribute of interest is to be measured, R₁,…R_n are empirical relations in E, and o₁,…,o_m are binary operations on the real-world entities that must also be measured. The empirical relational system is a formalisation of our intuitive understanding, and is thus expressed only in a subjective manner, i.e., only qualitative statements are made. In this system a relation represents intuitive knowledge about an attribute of the entities that we measure. This intuitive knowledge is also characterised as a viewpoint (Chidamber and Kemerer, 1994). For example, a designer understands that a class of objects with more member functions is generally more complex¹ than a class with fewer methods, which leads to the viewpoint that an increase in the number of member functions generally increases the complexity. Although a viewpoint is identical to an empirical relation (Chidamber and Kemerer, 1994), we use the term “viewpoint” instead of “empirical relation” to be consistent with (Chidamber and Kemerer, 1994). Similar to an empirical relational system a formal relational system is defined as $\text{B}\text{=(B,S}{}_1\text{,…S}{}_{\mathrm{n}}\text{,p}{}_1\text{,…,p}{}_{\mathrm{m}}\text{)}$, where B is a non-empty set of formal objects such as numbers, S₁,…S_n, are relations in B and p₁,…,p_m, are closed binary operations.

A metric (or measure) is a mapping μ:E→B. This mapping, according to the representational theory of measurement must define a scale. The triple ($\text{E}\text{,}\text{B}\text{,μ}$) is a scale iff for all i,j and all e₁,…e_n,f,g∈E, we have R_i(e₁,…e_n)⇔S_i(s₁,…s_n), and μ(fo_jg)=μ(f)p_jμ(g) (Briand et al., 1995). What this implies is that a metric maps entities to numbers (formal objects) such that the meaning of all empirical relations is preserved. For example, if program e₁ is “more complex than” program e₂ and μ is claimed to be a real-valued complexity measure, then in the numerical system μ(e₁) should be greater than μ(e₂) with the relation more complex than being mapped to the relation ` > '. This requirement is also termed as the representation condition. The definition of scale also implies that measured values cannot be used or combined arbitrarily, they must be used meaningfully and combined through admissible mathematical operations only.

It is amply clear that a metric must be defined, systematically, in a manner that takes into account all the requirements of the representational theory of measurement. The steps that must be followed are summarised in Fig. 1 (Fenton and Pfleeger, 1996). Hitz and Montazeri (1996) present this sequence more comprehensively. In Fig. 1, the last step emphasises that numerical relations must preserve and are preserved by empirical relations or viewpoints. It is the preservation of viewpoints in numerical relations that we have tried to empirically validate for the metrics DIT and NOC, and which we also refer to as viewpoint representation validation.

An attempt to validate object-oriented metrics using measurement theory criteria (Roberts, 1979, Fenton and Pfleeger, 1996, Whitmire, 1997) and other formal properties, e.g., those of Weyuker (1988), has been reported in the literature. Measurement theory has been used by Hitz and Montazeri (1996) to analyse the C&K suite of metrics, by Harrison and Counsell (1998) to analyse the MOOD set of metrics, and by Whitmire (1997) to propose and analyse object-oriented metrics based on “category theory”. Chidamber and Kemerer (1994), and Gursaran and Roy (2001) have used Weyuker's properties to analyse object-oriented metrics, and Churcher and Shepperd (1995) have used program examples to show that the metrics in the C&K suite have not been defined rigorously. Taking a different approach to defining metrics and validating them, Briand et al. (1999) have derived object-based measures, for cohesion and coupling, from an experimental goal and several experimental hypotheses, and have empirically investigated their relation to fault-proneness on three large projects. Besides these efforts, empirical conditions behind object-oriented metrics have been investigated by Zuse and Fetcke (1995), and Whitmire (1997), and general approaches to validation of software metrics and prediction models, which can be used in the object-oriented case, have been proposed by Schneidewind (1994) and Kitchenham et al. (1995).

Much of the validation based on measurement theory is theoretical. But, it is also important to attempt to empirically establish that the representation condition is satisfied. This, however, is difficult for object-oriented metrics as the theoretical constructs pertaining to the measured attributes of object-oriented software products have not yet been formalised to an extent that they can be used to analytically establish relations, for instance, in a set of classes, and then used to validate the representation condition. Keeping this in mind we have taken an approach that is simple and yet effective in dealing with the problems of empirical validation of representation. This approach is described in the following section.

In Section 2 we describe the conceptual model behind the design of the empirical study that was carried out by us. The design of the experiment is discussed in Section 3, and in Section 4 we present the results of the experiment. In Section 5 we discuss the results presented in Section 4 and also introduce “vectors of metrics” as an alternate strategy for collecting and presenting metrics. Finally in Section 6 we present our conclusions.