High Average-Utility Itemset Sampling Under Length Constraints

Abstract

High Utility Itemset extraction algorithms are methods for discovering knowledge in a database where the items are weighted. Their usefulness has been widely demonstrated in many real world applications. The traditional algorithms return the set of all patterns with a utility above a minimum utility threshold which is difficult to fix, while top-k algorithms tend to lack of diversity in the produced patterns. We propose an algorithm named $\textsc {HAISampler}$ to sample itemsets where each itemset is drawn with a probability proportional to its average-utility in the database and under length constraints to avoid the long and rare itemsets with low weighted items. The originality of our method stems from the fact that it combines length constraints with qualitative and quantitative utilities. Experiments show that $\textsc {HAISampler}$ extracts thousands of high average-utility patterns in a few seconds from different databases.

A Appendix (Proof of Theoretical Results)

Proof

(Property 1). Let’s start by showing that $\omega ^-_{\ell }(t [i], t)= \sum _{\star \in \{+,-\}}\omega ^\star _{\ell }(t [i+1], t)$. By definition, $\omega ^-_{\ell }(t [i], t)$ is the sum of the utilities of the set of patterns of length $\ell$ in $t ^i$, $\omega ^-_{\ell }(t [i], t)=\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell } \texttt {uOcc} (\varphi ,t)$. This set can be split into two parts: the one that contains the patterns starting with the item $t [i+1]$ whose sum of their utilities is equal to $\omega ^+_{\ell }(t [i+1], t)$ by definition, and the one that contains the patterns not starting with $t [i+1]$ and whose sum of their utilities is equal to $\omega ^-_{\ell }(t [i+1], t)$. It implies that

$$\begin{aligned} \sum \nolimits _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell } \texttt {uOcc} (\varphi ,t)=\omega ^+_{\ell }(t [i+1], t)+\omega ^-_{\ell }(t [i+1], t)=\sum \nolimits _{\star \in \{+,-\}}\omega ^\star _{\ell }(t [i+1], t). \end{aligned}$$

(1)

Let’s show that $\omega ^+_{\ell }(t [i], t)= \omega _{1}^+(t [i], t)\times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell -1\end{array}}\right) + \sum _{\star \in \{+,-\}}\omega ^\star _{\ell -1}(t [i+1], t)$. We know by definition that $\omega ^+_{\ell }(t [i], t)$ is the sum of utilities of itemsets of length $\ell$ in $t ^{i-1}$ which start with $t [i]$ following the total order relation $>_\mathcal{I}$. Formally, we have: $\omega ^+_{\ell }(t [i], t)=\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\{t [i]\} \cup \varphi ,t)$. But $\texttt {uOcc} (\{t [i]\} \cup \varphi ,t)= \texttt {uOcc} (\{t [i]\},t) + \texttt {uOcc} (\varphi ,t)$ by definition. Then, $\omega ^+_{\ell }(t [i], t)=\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \left( \texttt {uOcc} (\{t [i]\},t) + \texttt {uOcc} (\varphi ,t) \right)$. This implies: $\omega ^+_{\ell }(t [i], t)=\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\{t [i]\},t) + \sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\varphi ,t)$. However, we know on the one hand that $\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\{t [i]\},t)=\texttt {uOcc} (\{t [i]\},t) \times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell -1\end{array}}\right)$ and $\texttt {uOcc} (\{t [i]\},t)=\omega ^+_{1}(t [i], t)$ by definition, so $\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1}\texttt {uOcc} (\{t [i]\},t)= \omega ^+_{1}(t [i], t) \times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell -1\end{array}}\right)$. On the other hand, $\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\varphi ,t)$ is the sum of utilities of the set of patterns of length $\ell -1$ in the transaction $t ^i$. From (1), we can also say that $\sum _{\varphi \subseteq t ^i \wedge {|{\varphi }|} =\ell -1} \texttt {uOcc} (\varphi ,t)=\sum _{\star \in \{+,-\}}\omega ^\star _{\ell -1}(t [i+1], t)$. Then we have: $\omega ^+_{\ell }(t [i], t)=\omega _{1}(t [i], t)\times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell -1\end{array}}\right) + \sum _{\star \in \{+,-\}}\omega ^\star _{\ell -1}(t [i+1], t)$. Hence the result. $\square$

Proof

(Property 2). By definition, the weight of the transaction $t$ is the sum of the average-utilities of the pattern occurrences it contains. According to Property 1, the weight of the transaction $t$ under the minimum $m$ and maximum $M$ length constraints is nothing more than the sum of the average-utilities of pattern occurrences that start with the item $t [1]$ and respect the imposed length constraints, $\sum _{\ell =m}^{M}(\frac{1}{\ell } \times \omega ^+_{\ell }(t [1], t))$, and that of the patterns that do not start with the item $t [1]$ but respect the length constraints, $\sum _{\ell =m}^{M}(\frac{1}{\ell } \times \omega ^-_{\ell }(t [1], t))$. However, we know that $\sum _{\ell =m}^{M}(\frac{1}{\ell } \times \omega ^+_{\ell }(t [1], t))+\sum _{\ell =m}^{M}(\frac{1}{\ell } \times \omega ^-_{\ell }(t [1], t))=\sum _{\ell =m}^{M}\frac{1}{\ell } \times \left( \omega ^+_{\ell }(t [1], t) + \omega ^-_{\ell }(t [1], t) \right) .$ Hence the result. $\square$

Proof

(Lemma 1). By definition, the probability to draw the item $t [i]$ of the transaction $t$ after having drawing from it $\ell -\ell '$ items and store them in $\varphi$ is nothing more than the probability of drawing a pattern that begins with $\varphi \cup \{t [i]\}$, according to the order relation $>_\mathcal{I}$, among the set of patterns that start with $\varphi$. On the one hand, we know that the set of patterns of length $\ell$ that start with $\varphi \cup t [i]$ is defined by $\{\varphi ''\subseteq t:(\varphi ''=\varphi \cup \{t [i]\}\cup \varphi ')(\varphi '\subseteq t ^i)({|{\varphi '}|} =\ell '-1)\}$. The sum of utilities of the patterns of this set is equal to $\sum _{\varphi '\subseteq t ^i\wedge {|{\varphi '}|} =\ell '-1}\texttt {uOcc} (\varphi \cup \{t [i]\}\cup \varphi ', t)$. On the other hand, we know that the set of patterns of length $\ell$ that start with $\varphi$ is defined by $\{\varphi ''\subseteq t:(\varphi ''=\varphi \cup \varphi ')(\varphi '\subseteq t ^{i-1})({|{\varphi '}|} =\ell ')\}$. The sum of utilities of the patterns of this set is equal to $\sum _{\varphi '\subseteq t ^{i-1}\wedge {|{\varphi '}|} =\ell '}\texttt {uOcc} (\varphi \cup \varphi ', t)$. So $\mathbb {P} _\ell ^t (t [i]|\varphi , \ell ')= \frac{\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \texttt {uOcc} (\varphi \cup \{t [i]\}\cup \varphi ',t)}{\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '} \texttt {uOcc} (\varphi \cup \varphi ',t)}$. Hence the result. $\square$

Proof

(Property 3). From Lemma 1, we have:

$\mathbb {P} _\ell ^t (t [i]|\varphi , \ell ')= \frac{\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \texttt {uOcc} (\varphi \cup \{t [i]\}\cup \varphi ',t)}{\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '} \texttt {uOcc} (\varphi \cup \varphi ',t)}$. First, by definition we have $\texttt {uOcc} (\varphi \cup \{t [i]\}\cup \varphi ',t)=\texttt {uOcc} (\varphi ,t)+\texttt {uOcc} (\{t [i]\}\cup \varphi ',t)$. Let $z_i=\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \texttt {uOcc} (\varphi \cup \{t [i]\}\cup \varphi ',t)$. It implies that $z_i=\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \left( \texttt {uOcc} (\varphi ,t)+\texttt {uOcc} (\{t [i]\}\cup \varphi ',t) \right)$. Then we have: $z_i = \sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \texttt {uOcc} (\varphi ,t)+ \sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1}\texttt {uOcc} (\{t [i]\}\cup \varphi ',t)$. But $\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1} \texttt {uOcc} (\varphi ,t)=\texttt {uOcc} (\varphi ,t)\times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell '-1\end{array}}\right)$ and $\sum _{\varphi '\subseteq t ^i \wedge {|{\varphi '}|} =\ell '-1}\texttt {uOcc} (\{t [i]\}\cup \varphi ',t)=\omega _{\ell '}^+(t [i],t)$ by definition. Then $z_i = \texttt {uOcc} (\varphi ,t)\times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell '-1\end{array}}\right) +\omega _{\ell '}^+(t [i],t)$. We also know that $\texttt {uOcc} (\varphi ,t)=\sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t)$. So, $z_i = \left( \sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t) \right) \times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell '-1\end{array}}\right) +\omega _{\ell '}^+(t [i],t)$. Second, we have $\texttt {uOcc} (\varphi \cup \varphi ',t)=\texttt {uOcc} (\varphi ,t)+\texttt {uOcc} (\varphi ',t)$. By setting $Z_i=\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '} \texttt {uOcc} (\varphi \cup \varphi ',t)$, we get then $Z_i=\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '} \texttt {uOcc} (\varphi ,t)+ \sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '}\texttt {uOcc} (\varphi ',t)$. But $\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '} \texttt {uOcc} (\varphi ,t)= \texttt {uOcc} (\varphi ,t)\times \left( {\begin{array}{c}{|{t ^{i-1}}|} \\ \ell '\end{array}}\right) =\left( \sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t) \right) \times \left( {\begin{array}{c}{|{t ^{i-1}}|} \\ \ell '\end{array}}\right)$ et $\sum _{\varphi '\subseteq t ^{i-1} \wedge {|{\varphi '}|} =\ell '}\texttt {uOcc} (\varphi ',t)=\sum _{\star \in \{+,-\}}\omega ^\star _{\ell '}(t [i],t)$, so $Z_i=\left( \sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t) \right) \times \left( {\begin{array}{c}{|{t ^{i-1}}|} \\ \ell '\end{array}}\right) + \sum _{\star \in \{+,-\}}\omega ^\star _{\ell '}(t [i],t).$

Finally, $\mathbb {P} _\ell ^t (t [i]|\varphi , \ell ')=\frac{z_i}{Z_i}=\frac{\left( \sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t) \right) \times \left( {\begin{array}{c}{|{t ^i}|} \\ \ell '-1\end{array}}\right) +\omega _{\ell '}^+(t [i],t)}{\left( \sum _{k<i\wedge t [k]\in \varphi }\omega _1^+(t [k], t) \right) \times \left( {\begin{array}{c}{|{t ^{i-1}}|} \\ \ell '\end{array}}\right) + \sum _{\star \in \{+,-\}}\omega ^\star _{\ell '}(t [i],t)}$. $\square$

Proof

(Property 4). Let $m$ be the minimum and $M$ the maximum length constraints, the probability of drawing the pattern $\varphi$ of length $\ell$ in the database $\mathcal{D}$ denoted by $\mathbb {P} _{[m..M ]}(\varphi ,\mathcal{D})$, and Z a normalization constant defined by $Z=\sum _{\varphi '\in \mathcal{L} (\mathcal{D})}u^{avg} _{[m..M ]}(\varphi ',\mathcal{D})$. We know that : $\mathbb {P} _{[m..M ]}(\varphi ,\mathcal{D})=\sum _{(j,t) \in \mathcal{D}}\left( \mathbb {P} _{[m..M ]}(t _j,\mathcal{D}) \times \mathbb {P} _{[m..M ]}(\varphi , t _j) \right)$. But $\mathbb {P} _{[m..M ]}(t _j,\mathcal{D})=\frac{\omega ^{avgU} _{[m..M ]}(t _j)}{Z}$, then

$$\begin{aligned} \mathbb {P} _{[m..M ]}(\varphi ,\mathcal{D})=\sum \nolimits _{(j,t) \in \mathcal{D}}\left( \frac{\omega ^{avgU} _{[m..M ]}(t _j)}{Z} \times \mathbb {P} _{[m..M ]}(\varphi , t _j) \right) . \end{aligned}$$

(2)

We also know that:

$$\begin{aligned} \mathbb {P} _{[m..M ]}(\varphi , t _j)=\mathbb {P} _{[m..M ]}(\ell |t _j)\times \mathbb {P} _{[m..M ]}^{t _j}(\varphi |\ell ). \end{aligned}$$

(3)

$\mathbb {P} _{[m..M ]}(\ell |t _j)=\frac{\omega ^{avgU} _{[\ell ..\ell ]}(t _j)}{\omega ^{avgU} _{[m..M ]}(t _j)}$ and $\mathbb {P} _{[m..M ]}^{t _j}(\varphi |\ell )=\frac{\texttt {uOcc} (\varphi ,t _j)}{\omega ^{avgU} _{[\ell ..\ell ]}(t _j)\times \ell }$ then by substituting the two terms in (3), $\mathbb {P} _{[m..M ]}(\varphi , t _j)=\frac{\omega ^{avgU} _{[\ell ..\ell ]}(t _j)}{\omega ^{avgU} _{[m..M ]}(t _j)} \times \frac{\texttt {uOcc} (\varphi ,t _j)}{\omega ^{avgU} _{[\ell ..\ell ]}(t _j)\times \ell }=\frac{\texttt {uOcc} (\varphi ,t _j)}{\omega ^{avgU} _{[m..M ]}(t _j)\times \ell }$.

Now, if we replace $\mathbb {P} _{[m..M ]}(\varphi , t _j)$ in (2) by its last expression, we get: $\mathbb {P} _{[m..M ]}(\varphi ,\mathcal{D})=\sum _{(j,t) \in \mathcal{D}}\left( \frac{\omega ^{avgU} _{[m..M ]}(t _j)}{Z} \times \frac{\texttt {uOcc} (\varphi ,t _j)}{\omega ^{avgU} _{[m..M ]}(t _j)\times \ell }\right) =\frac{1}{Z} \times \frac{\sum _{(j,t) \in \mathcal{D}} \texttt {uOcc} (\varphi ,t _j)}{\ell }$. But by definition, we have $\frac{\sum _{(j,t) \in \mathcal{D}} \texttt {uOcc} (\varphi ,t _j)}{\ell }=u^{avg} _{[m..M ]}(\varphi ,\mathcal{D})$, so $\mathbb {P} _{[m..M ]}(\varphi ,\mathcal{D})=\frac{u^{avg} _{[m..M ]}(\varphi ,\mathcal{D})}{Z}.$ Hence the result. $\square$