PARAS$^{\mathrm{c}}$: a parameter space-driven approach for complete association rule mining

Abstract

To enable efficient association rule mining, existing techniques prestore intermediate results as itemsets. However, the actual rule generation is still performed at query-time. The response time thus tends to remain unacceptably long for interactive mining, especially when rule redundancy resolution is required. Further, the widespread restriction to only support positive rules can miss important insights and lead to misleading results. For this reason, the discovery of both negative and positive rules, which can be extremely revealing, is important. Unfortunately, the generation of negative rules slows down the mining process even further. To tackle these shortcomings, we introduce the parameter space model, called ${\textbf {PARAS}}^{{\textbf {c}}}$. ${\textbf {PARAS}}^{{\textbf {c}}}$ enables efficient mining of complete rules, i.e., both positive and negative rules, by precomputing and compactly maintaining the final rulesets. The ${\textbf {PARAS}}^{{\textbf {c}}}$ model is based on the stable region abstractions that form the coarse granularity ruleset space for managing complete rules. Based on new insights into the redundancy relationships among complete rules, ${\textbf {PARAS}}^{{\textbf {c}}}$ establishes a surprisingly compact representation of complex redundancy relationships while enabling efficient redundancy resolution for complete rules at query-time. ${\textbf {PARAS}}^{{\textbf {c}}}$ supports novel classes of exploratory queries that can be answered near real time. Our experimental evaluation demonstrates that ${\textbf {PARAS}}^{{\textbf {c}}}$ achieves 2–5 orders of magnitude improvement over existing techniques in rule mining.

Appendix A

1.1 Proof to Lemma 8

Proof

Given a positive rule ${\mathcal {R}}$ contains only positive items, each simple dominating rule of ${\mathcal {R}}$ contains the same positive items as ${\mathcal {R}}$ (Definition 12). Thus it is also positive. Therefore, even when the scope of full-spectrum rule mining includes both positive and negative rules, a positive rule is only simply dominated by positive rules. $\square$

1.2 Proof to Lemma 9

Proof

Given a negative rule ${\mathcal {R}}$ contains at least one negative item, each simple dominating rule of ${\mathcal {R}}$ contains the same itemset (including the negative item(s)) as ${\mathcal {R}}$ (Definition 12). It is thus negative. Therefore, a negative rule is only simply dominated by negative rules. $\square$

1.3 Proof of Lemma 10

Proof

Given rule ${\mathcal {R}}$: ($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$), a 1-antecedent simple dominating rule⁷ is the rule using 1 item out of the set ($\hbox {A}_{1}$:$\hbox {A}_{n}$) as antecedent, the remaining (n-1) items ($\hbox {A}_{v}$:$\hbox {A}_{w}$) along with ($\hbox {C}_{1}$:$\hbox {C}_{m}$) as consequent. The total number of such 1-antecedent simple dominating rules for (($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$)) is up to $\left( ^{n}_{1}\right)$.⁸ Similarly, the number of 2-antecedent simple dominating rules is up to $\left( ^{n}_{2}\right)$ ${}^{8}$ . Therefore, the number of simple dominating rules for (($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$)) is up to $\sum _{i=1}^{n-1}\left( ^{n}_{i} \right)$ = 2$^{n}$-2. $\square$

1.4 Proof of Lemma 13

Proof

A positive rule ${\mathcal {R}}$ is generated from an itemset $X_{j}$ consisting of only positive items. Each rule that strictly dominates ${\mathcal {R}}$ is constructed from an itemset $X_{i}$, with $X_{i}\supset X_{j}$. That is, $X_{i}$ contains all items in $X_{j}$ plus some additional items beyond $X_{j}$ (Definition 13). If the additional items are all positive, then the formed strict dominating rule is positive, else it is negative. Therefore, a positive rule can be strictly dominated by both positive and negative rules. $\square$

1.5 Proof of Lemma 13

Proof

For a negative rule ${\mathcal {R}}$ generated from itemset $X_{j}$ with at least one negative item, each of its strict dominating rules is generated from a superset of $X_{j}$ (Definition 13). Each strict dominating rules of ${\mathcal {R}}$ thus also contains the same negative item(s). It is thus a negative rule. Thus, a negative rule can only be strictly dominated by negative rules. $\square$

1.6 Proof of Lemma 14

Proof

Given a strict dominated rule ${\mathcal {R}}$: ($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$), a 1-antecedent strict dominating rule is the rule using 1 item out of the set ($\hbox {A}_{1}$:$\hbox {A}_{n}$) as antecedent and adding w ($1 \le w \le e$) items from e additional items to the consequent. In the context of positive rules, for each additional item, there are 2 possibilities (positive form or not chosen). For a 1-antecedent strict dominating rule, there are $2^{e}$ possibilities for adding items from e additional items to the consequent. The total of number of such 1-antecedent strict dominating rules is up to $\left( ^{n}_{1}\right) \times 2^{e}$. Similarly, the number of 2-antecedent strict dominating rules is up to $\left( ^{n}_{2}\right) \times 2^{e}$. Therefore, the number of strict dominating rules for ($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$) is up to $2^{n+e}$.

In the context of complete rules, for each additional item, there are 3 possibilities (positive form, negative form, not chosen). Therefore, there are $3^{e}$ possibilities for adding items from e additional items to the consequent. The number of strict dominating rules for ($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$) is up to $2^{n}\times 3^{e}$. $\square$

1.7 Proof of Lemma 15

Proof

Within the strict dominating rule set $\{{\mathcal {R}}^{\gg _{str}}\}$, the strict dominating rules formed by adding one item $C_{h}$ to the consequent have larger or equal support value than the strict dominating rules formed by adding the same item $C_{h}$ plus one more item $C_{g}$ to the consequent, with $C_{h}$ and $C_{g}$ each are a single item out of the set {$\hbox {C}_{m+1}$:$\hbox {C}_{m+e}$}. That is, adding only one item into consequent of ${\mathcal {R}}$ tends to have the larger support value. Let us assume adding item $C_{q}$ incurs the largest support value, namely, the rules based on the underlying itemset (($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{q}$) have the highest support value among all other rules in the strict dominating rule set $\{{\mathcal {R}}^{\gg _{str}}\}$. In fact, we note that these strict dominating rules composed by all items from the itemset (($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\cup$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$) $\cup$ $\hbox {C}_{q}$) have the same support value. They could have 1 to n items in the antecedent, namely, a subset of ($\hbox {A}_{1}$:$\hbox {A}_{n}$) as antecedent. The strict dominating rules with the largest confidence value would have all n items in its antecedent because its confidence equals Supp(($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\cup$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$) $\cup$ $\hbox {C}_{q}$) / Supp($\hbox {A}_{1}$:$\hbox {A}_{n}$). The denominator is the smallest when all n items are included. Thus the confidence value is the highest for the rule that is composed by items ($A_{1}$:$A_{n}$) as antecedent and ($C_{1}$:$C_{m}$) $\cup$ $C_{q}$ as consequent. The support and confidence value of rule (($A_{1}$:$A_{n}$) $\rightarrow$ ($C_{1}$:$C_{m}$) $\cup$ $C_{q}$) is the strict dominating location $\ell ^{\gg _{str}}$ of ${\mathcal {R}}$. $\square$

1.8 Proof of Theorem 1

Proof

Since the result rule set $\{{\mathcal {R}}\}$ is derived from the query using parameter values (minsup, minconf), the coordinate of each rule ${\mathcal {R}}$ in the result set $\{{\mathcal {R}}\}$ must be larger than the user provided parameter (minsup, minconf), namely, minsupp $\le$ ${\mathcal {R}}.supp$ and minconf $\le$ ${\mathcal {R}}.conf$. Three possible scenarios are present below:

Case 1: The rule with the highest support and confidence value in the strict dominating rule set of ${\mathcal {R}}$ is located in the strict dominating location of ${\mathcal {R}}$ by Lemma 15. If both of the user given parameter settings are lower than or equal to the strict dominating location’s coordinate, namely, $minsupp\le$ $\ell ^{\gg _{str}}$.supp AND $minconf\le$ $\ell ^{\gg _{str}}$.conf, then one or more strict dominating rules of ${\mathcal {R}}$ are valid. This would make ${\mathcal {R}}$ redundant.

Case 2: The rule with the highest confidence value in the simple dominating rule set of ${\mathcal {R}}_{j}$ is located in the simple dominating location of ${\mathcal {R}}_{j}$ by Lemma 11. If both of the user given parameter settings are lower than or equal to the simple dominating location’s coordinate, namely, $minsupp\le \ell ^{\gg _{sim}}$.supp AND $minconf\le$ $\ell ^{\gg _{sim}}$.conf, then one or more simple dominating rules of ${\mathcal {R}}$ are valid. This would make ${\mathcal {R}}$ redundant. Note that Case 1 and Case 2 are partial overlapping, meaning the user given parameter setting could be lower than both the simple and strict dominating location.

Case 3: If the minsupp parameter is greater than the strict dominating location’s support and the minconf parameter is greater than the simple dominating location’s confidence, namely, minsupp > $\ell ^{\gg _{str}}.supp$ AND minconf > $\ell ^{\gg _{sim}}.conf$, then none of dominating rules of ${\mathcal {R}}$ is valid. Thus ${\mathcal {R}}$ is not redundant.

$\square$

1.9 Proof of Theorem 2

Proof

We first assume that all top simple dominating rules are valid. We prove that it is necessary and sufficient to search only the top simple dominating rules.

Scenario 1: All top simple dominating rules are valid. “Necessary”: To find the top simple dominating location of ${\mathcal {R}}$, it is necessary to consider all the (n-1)-antecedent simple dominating rules as any one of them can be the simple dominating location (Lemma 11) depending on each of these simple dominating rules’ confidence.

“Sufficient”: We further prove the sufficiency as follows. For a rule ${\mathcal {R}}$: ($\hbox {A}_{1}$:$\hbox {A}_{n}$) $\rightarrow$ ($\hbox {C}_{1}$:$\hbox {C}_{m}$) with n antecedents, all rules in the set of simple dominating rules $\{{\mathcal {R}}^{\gg _{sim}}\}$ have the same support value as rule ${\mathcal {R}}$ by Lemma 2. However, their confidence values may differ. Next, we argue that the (n-1)-antecedent dominating rules have the largest confidence among all 2$^{n}$-2 simple dominating rules. Recall that the simple dominating rules conform to the template ((($A_{1}$:$A_{n}$)-($A_{v}$:$A_{w}$)) $\rightarrow$ (($A_{v}$:$A_{w}$) $\cup$ ($C_{1}$:$C_{m}$))). Thus the confidence for such a rule is given by the formula $\frac{S((A_{1}:A_{n})\cup (C_{1}:C_{m}))}{S((A_{1}:A_{n})-(A_{v}:A_{w}))}$. The numerator is the same for all simple dominating rules, while the denominators differ. S(($A_{1}$:$A_{n}$)-$A_{i}$), where $A_{i}$ is a single item, is the smallest denominator compared to all other rules that remove $A_{i}$ in addition to any other $A_{j}(1 \le j \le n)$. This then makes the confidence of rules ((($A_{1}$:$A_{n}$)-$A_{i}$) $\rightarrow$ ($A_{i}\cup$($C_{1}$:$C_{m}$))) the highest among all simple dominating rules. Hence, the confidence of all (n-1)-antecedent simple dominating rules (aka top simple dominating rules) will be the highest. It is thus sufficient to only consider them instead of all 2$^{n}$-2 simple dominating rules.

Scenario 2: One or more top simple dominating rules are invalid.

We then consider the scenario that one or more top simple dominating rules are invalid. Suppose rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ is one of the top simple dominating rules of rule ${\mathcal {R}}$. Suppose rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ is an invalid rule, it could be invalid for two possible reasons: 1), it has insufficient support or confidence. 2), it has an insufficient lift.

For 1), we simply remove rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ because its insufficient support or confidence indicates that all its simple dominating rules do not have sufficient support or confidence by Lem. 2. For 2), although rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ is invalid due to insufficient lift, it is still possible that its simple dominating rules have sufficient lift. Therefore, one or more simple dominating rules of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ could be valid. These rules may have a higher confidence value than all valid simple dominating rules of ${\mathcal {R}}$. In this case, we cannot just remove rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$. Instead, we need to look for a valid simple dominating rule of rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ which has the largest confidence value among all simple dominating rules of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$. In other words, we need the simple dominating location of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ to participate in the comparisons with all valid top simple dominating rules of rule ${\mathcal {R}}$ because the simple dominating location of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ represents the largest support and confidence values of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$’s simple dominating rules. Below we prove that it is necessary and sufficient to search only the top simple dominating rules in scenario 2.

“Necessary”: same as Scenario 1.

“Sufficient”: For the simple dominating rule set $\{{\mathcal {R}}^{\gg _{sim}}\}$ of rule ${\mathcal {R}}$, suppose 1) rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ $\in$ $\{{\mathcal {R}}^{\gg _{sim}}\}$; 2) rule ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ is a top simple dominating rule of rule ${\mathcal {R}}$; 3) ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ is an invalid rule as it has sufficient support/confidence yet insufficient lift. The simple dominating location of ${{\mathcal {R}}^{\gg _{sim}}}_{j}$ saves the largest support and confidence value which ${{\mathcal {R}}^{\gg _{sim}}}_{j}$’s simple dominating rules have. Meanwhile, each valid top simple dominating rule’s confidence value is greater than its respective simple dominating rules. Therefore, the simple dominating location(s) of invalid top simple dominating rule(s) and the support and confidence values of valid top simple dominating rules complete the candidate set for computing simple dominating location for ${\mathcal {R}}$. $\square$

1.10 Proof of Theorem 3

Proof

We first assume all top strict dominating rules are valid.

Scenario 1: All top strict dominating rules are valid.

“Necessary”: We first prove the necessity as below. For the strict dominating rules of ${\mathcal {R}}$ formed by including additional item $C_{h}$ beyond the items from ${\mathcal {R}}$ into the consequent, namely, the top strict dominating rules, it cannot be decided which of them has the highest support and confidence value without inspecting each. Thus, we need to examine every top strict dominating rule of ${\mathcal {R}}$ to determine the strict dominating location.

“Sufficient”: We further prove the sufficiency as follows. For a rule ${\mathcal {R}}$: ($A_{1}$:$A_{n}$) $\rightarrow$ ($C_{1}$:$C_{m}$), the rules in the strict dominating rule set $\{{\mathcal {R}}^{\gg _{str}}\}$ have the underlying itemset containing all items ($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) from rule ${\mathcal {R}}$ and a subset of items from ($C_{h}$:$C_{h+e}$) beyond rule ${\mathcal {R}}$. Among them the rule formed by adding only one item from ($C_{h}$:$C_{h+e}$) tends to have a larger support value than the rule formed by adding more than one item, namely, Supp(($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{h}$) $\ge$ Supp(($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{h}$ $\cup$ $C_{h'}$ $\cup$ $\ldots )$. Assume here that adding item $C_{h}$ to rule ${\mathcal {R}}$ would bring us the largest support value compared to choosing another item. Then the strict dominating rules composed of all items from the itemset (($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{h}$) have the same support value. They could have 1 to n items as the antecedent, namely, a subset of ($A_{1}$:$A_{n}$) as antecedent. This leads to the fact that each rule may have different confidence values. The strict dominating rules with the largest confidence value would have all n items in the antecedent because its confidence equals Supp(($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{h}$) / Supp($A_{1}$:$A_{n}$). The denominator is the smallest when all n items are included. Thus the confidence value is the largest for the rule that is composed of the itemset (($A_{1}$:$A_{n}$) $\cup$ ($C_{1}$:$C_{m}$) $\cup$ $C_{h}$). Thus, it is necessary and sufficient to consider all top strict dominating rules.

Scenario 2: One or more top strict dominating rules are invalid.

“Necessary”: Same as Scenario 1.

“Sufficient”: For the strict dominating rule set $\{{\mathcal {R}}^{\gg _{str}}\}$ of rule ${\mathcal {R}}$, suppose 1) rule ${{\mathcal {R}}^{\gg _{str}}}_{k}$ $\in$ $\{{\mathcal {R}}^{\gg _{str}}\}$; 2) rule ${{\mathcal {R}}^{\gg _{str}}}_{k}$ is a top strict dominating rule of rule ${\mathcal {R}}$; 3) ${{\mathcal {R}}^{\gg _{str}}}_{k}$ is an invalid rule as it has sufficient support/confidence yet insufficient lift. The strict dominating location of ${{\mathcal {R}}^{\gg _{str}}}_{k}$ records the largest support and confidence value that ${{\mathcal {R}}^{\gg _{str}}}_{k}$’s simple dominating rules and strict dominating rules have. Meanwhile, each valid top strict dominating rule’s support and confidence values are larger than its respective simple/strict dominating rules. Thus, the strict dominating location(s) of invalid top strict dominating rule(s) and the (support, confidence) values of valid top strict dominating rule(s) comprise the strict dominating location candidate set for computing strict dominating location for ${\mathcal {R}}$. The largest (support, confidence) among the strict dominating location candidate set is the largest values among all strict dominating rule’s location. It is thus sufficient to only search the top strict dominating rules. $\square$