Aggregating Association Rules to Improve Change Recommendation

Abstract

As the complexity of software systems grows, it becomes increasingly difficult for developers to be aware of all the dependencies that exist between artifacts (e.g., files or methods) of a system. Change recommendation has been proposed as a technique to overcome this problem, as it suggests to a developer relevant source-code artifacts related to her changes. Association rule mining has shown promise in deriving such recommendations by uncovering relevant patterns in the system’s change history. The strength of the mined association rules is captured using a variety of interestingness measures. However, state-of-the-art recommendation engines typically use only the rule with the highest interestingness value when more than one rule applies. In contrast, we argue that when multiple rules apply, this indicates collective evidence, and aggregating those rules (and their evidence) will lead to more accurate change recommendation. To investigate this hypothesis we conduct a large empirical study of 15 open source software systems and two systems from our industry partners. We evaluate association rule aggregation using four variants of the change history for each system studied, enabling us to compare two different levels of granularity in two different scenarios. Furthermore, we study 40 interestingness measures using the rules produced by two different mining algorithms. The results show that (1) between 13 and 90% of change recommendations can be improved by rule aggregation, (2) rule aggregation almost always improves change recommendation for both algorithms and all measures, and (3) fine-grained histories benefit more from rule aggregation.

Appendix:

1.1 A Proofs

Within this section we formally prove that DCG and HCG satisfies the properties of Definition 4. As expressed earlier, we have limited our study of aggregation functions to positive values. We leave out the proof for CG as it is simply the algebraic sum and therefore naturally satisfies all properties of Definition 4.

1.1.1 A.1 Proof for Discounted Cumulative Gain

Theorem 1

DCG (Definition 6) satisfies the properties in Definition 4 for non-negative values of an interestingness measure M.

Proof

Let M be an interestingness measure, R be a setof rules with non-negative interestingness values, andV = [v ₁,…,v _n ]be an ordered list of interestingness values of rules in R for measure M, such that∀i < j.v _i ≥ v _j .Let r be an arbitrary rule in R, and R ^′be equal to R ∖{r}.Then, there exists a v _k ∈ V such that M(r) = v _k .We define U = [u ₁,…,u _n− 1]as the ordered list of interestingness values of rules inR ^′for measure M. U is equal to V except thatv _k is removed from it, and we have ∀i < j.u _i ≥ u _j .Then:

$$\begin{array}{@{}rcl@{}} \forall 1 &\leq& i < k . v_{i} = u_{i} \end{array} $$

(6)

$$\begin{array}{@{}rcl@{}} \forall k < i &\leq& n . v_{i} = u_{i-1} \end{array} $$

(7)

The first property in Definition 4, which concerns Rs of size one, is trivial. To prove that the second property in Definition 4 holds for DCG, we compute the difference between the DCG of R and the DCG of R ^′ and show that forv _k > 0, this difference is positive,and for v _k = 0, this differenceis equal to zero. Let D C G(R, M) and D C G(R ^′, M) denote the DCGs of R and R ^′ for the interestingness measure M, respectively.

$$\begin{array}{@{}rcl@{}} DCG(R, M) - DCG(R^{\prime}, M) &=& \sum\limits_{i = 1}^{n} \frac{v_{i}}{log_{2}(i + 1)} - \sum\limits_{i = 1}^{n-1} \frac{u_{i}}{log_{2}(i + 1)} \\ &=& \sum\limits_{i = 1}^{k-1} \frac{v_{i} - v_{i}}{log_{2}(i + 1)} \\ &&+ \sum\limits_{i=k}^{n} \frac{v_{i}}{log_{2}(i + 1)} - \sum\limits_{i=k}^{n-1} \frac{u_{i}}{log_{2}(i + 1)} \\ &\overset{Eq 6,7}{=}& \sum\limits_{i=k}^{n-1} \frac{v_{i} - v_{i + 1}}{log_{2}(i + 1)} + \frac{v_{n}}{log_{2}(n + 1)} \end{array} $$

(8)

Note that both terms in the last line are always non-negative. Now, we consider two cases based on thevalue of v _k :

v _k > 0–Since v _k > 0,the two terms in (8) cannot be zero simultaneously. Because this requiresv _n = 0, and at thesame time ∀i ≥ k, v _i = v _i+ 1. Thelatter implies v _n = v _k > 0,which contradicts with the former. Therefore, in this case, there is always at least one positive termin (8). Thus:

$$ DCG(R, M) - DCG(R^{\prime}, M) > 0 $$

(9)

This proves that the second property in Definition 4 holds whenM(r) = v _k ispositive.

v _k = 0– In this caseD C G(R, M) − D C G(R ^′, M) = 0. This followsfrom the original assumption that the rules in V are ordered according to their absolute values. Therefore, in(8) all v _i s areequal to zero. □

1.2 A.2 Proof for Hyper Cumulative Gain

We now prove that HCG satisfies the monotonicity properties of Definition 4 for non-negative values of an interestingness measure M. We start by introducing a new operator, and two lemmas that are used in the proof.

Definition 13 (Correlative sum)

Let a ₁ and a ₂ be real numbers. For any nonzero real number b, we define the operator S _b as

$$a_{1} S_{b} a_{2} = a_{1} + \frac{b-a_{1}}{b} \cdot a_{2}. $$

Lemma 1 (Properties of correlative sum)

For any nonzero real number b, the correlative sum S _b is commutative, and associative.

Proof

S _b is commutative:

$$\begin{array}{@{}rcl@{}} a_{1} S_{b} a_{2} & = & a_{1} + \frac{b-a_{1}}{b} \cdot a_{2} \\ & = & \frac{a_{1} b + a_{2} b - a_{1} a_{2}}{b} \\ & = & a_{2} + \frac{b-a_{2}}{b} \cdot a_{1} \\ & = & a_{2} S_{b} a_{1} \end{array} $$

S _b isassociative:

$$\begin{array}{@{}rcl@{}} (a_{1} S_{b} a_{2}) S_{b} a_{3} & = & a_{1} + \frac{b-a_{1}}{b} \cdot a_{2} + \frac{b-(a_{1} + \frac{b-a_{1}}{b} \cdot a_{2})}{b} \cdot a_{3} \\ & = & \frac{a_{1} b + a_{2} b - a_{1}a_{2}} {b} + \frac{b-(\frac{a_{1} b + a_{2} b - a_{1}a_{2}}{b})}{b} \cdot a_{3} \\ & = & \frac{a_{1} b + a_{2} b - a_{1}a_{2}}{b} + \frac{b^{2}-(a_{1} b + a_{2} b - a_{1}a_{2} )}{b^{2}} \cdot a_{3} \\ & = & \frac{a_{1} b^{2} + a_{2} b^{2} - a_{1}a_{2}b + a_{3} b^{2} - a_{1}a_{3}b - a_{2}a_{3}b + a_{1}a_{2}a_{3}} {b^{2}} \\ & = & a_{1} + \frac{b-a_{1}}{b} \cdot \frac{a_{2} b + a_{3} b - a_{2}a_{3}} {b} \\ & = & a_{1} + \frac{b-a_{1}}{b} \cdot (a_{2} + \frac{b-a_{2}}{b} \cdot a_{3}) \\ & = & a_{1} S_{b} (a_{2} S_{b} a_{3}) \end{array} $$

□

An important implication of this lemma is that S _b can be applied to a sequence of numbers independent from the ordering of the elements in the sequence.

Lemma 2

For any nonzero real number b, let L = {l ₁, l ₂,...,l _n } be a sequence of real numbers. Let S _b (L) denote l ₁ S _b l ₂⋯l _n− 1 S _b l _n . Then

$$ S_{b}(L) = l_{1} + \sum\limits_{i = 2}^{n} \left( l_{i} \cdot \prod\limits_{j = 1}^{i-1}(1-\frac{l_{j}}{b})\right) $$

(10)

and for any given real number l, we have:

$$ S_{b}(L \cup \{l\}) = S_{b}(L) + l \cdot \prod\limits_{l_{j} \in L}(1-\frac{l_{j}}{b}). $$

(11)

Proof

The proof for both parts is straightforward after expanding the polynomials.□

An implication of (11) is that, for any sequence of real numbers L = {l ₁, l ₂,...,l _n }, and any arbitrary real number l in L, we have:

$$ S_{b}(L) - S_{b}(L \setminus \{l\}) = l \cdot \prod\limits_{l_{j} \in L \setminus \{l\}}(1-\frac{l_{j}}{b}) $$

(12)

Theorem 2

HCG (Definition 9) satisfies the properties of Definition 4 for non-negative values of an interestingness measure M.

Proof

Let M be a normalized interestingness measure with upper bound $b \in \mathbb {R} \setminus \{0\}$,R = r u l e s be a set of rules, and L _M = 〈M(r ₁), ⋯M(r _n )〉 be the sequence of non-negative interestingness values for the rules in R. HCG of$\mathscr {H}(R)$for M defined in Definition 9 can be rewritten as follows:

$$ HCG(\mathscr{H}(R), M) = \left( S_{b}(L_{M}) , m \right) $$

(13)

where m is given by

$$m = \vert\{r \in R | M(r) > 0\}\vert $$

The firstproperty in Definition 4, which concerns R s of size one, is again trivial. Let r be an arbitrary rule in R, and l denote M(r).To prove that the second property in Definition 4 holds for HCG, we compare the HCGs of R andR ∖{r}, and consider three casesbased on the value of M(r):

M(r) > 0– In this case, we need to show that the HCG of R is greater than the HCG ofR ∖{r}.

$$\begin{array}{@{}rcl@{}} HCG(\mathscr{H}(R), M) > HCG(\mathscr{H}(R \setminus \{r\}), M) & \overset{Eq 13}{\equiv}& \left( S_{b}(L_{M}) , m \right) > \left( S_{b}(L_{M} \setminus \{l\}) , m-1 \right) \\ & \overset{Def 10}{\equiv}& S_{b}(L_{M}) \geq S_{b}(L_{M} \setminus \{l\}) \end{array} $$

(14)

To show that inequality (14) holds we show thatS _b (L _M ) − S _b (L _M ∖{l})isnon-negative, which, according to (11), is equivalent to showing that

$$l \cdot \prod\limits_{l_{j} \in L_{M} \setminus \{l\}}(1-\frac{l_{j}}{b}) \geq 0 $$

This inequalityholds because l = M(r) > 0, andfor all l _j ∈ L _M ∖{l}the term$1-\frac {l_{j}}{b}$is non-negative (becausel _j is at most b). Sincem > m − 1, inequality (14) holds,completing the case for M(r) > 0.

M(r) = 0– In this case, we have toshow that the HCGs of R and R ∖{r}are equal.

$$ HCG(\mathscr{H}(R), M) = HCG(\mathscr{H}(R \setminus \{r\}), M) \overline{\equiv}{Eq 13} \left( S_{b}(L_{M}) , m \right) = \left( S_{b}(L_{M} \setminus \{l\}) , m \right) $$

To do so, we have to show that S _b (L _M ) = S _b (L _M ∖{l}).This is proven by forming S _b (L) − S _b (L _M ∖{l}),and replacing l with zero in the right-hand side of (11). Therefore, completing the proof.

So far, we have proven that HCG satisfies the properties in Definition 4 forinterestingness measures that have a finite upper bound. If the upper bound of aninterestingness measures is infinity, then like CG, HCG becomes the sum of interestingnessvalues of all rules in R. Therefore, it satisfies all the properties in Definition 4.□