Efficient algorithms to mine concise representations of frequent high utility occupancy patterns

Abstract

Identifying all frequent high utility occupancy itemsets (FHUOIs) in a quantitative transaction dataset is a new trend in data mining. By combining both factors of frequency and utility occupancy, these patterns are more suitable for several applications in the real world. These patterns not only reflect the interests of most users but also contribute a high proportion of the utility in supporting transactions. Nonetheless, the set of all discovered FHUOIs may be very large, especially for large and dense datasets or for low values of predefined minimum thresholds. For this reason, it is often quite challenging for users to analyze and use the obtained patterns. To address this issue, this paper proposes two novel algorithms named MaxCloFHUOIM and CloFHUOIM to extract compact representations of FHUOIs. The former is designed to simultaneously mine two concise representations of FHUOIs that consist of all closed FHUOIs and all maximal FHUOIs, whereas the latter only discovers the closed FHUOIs, which provide a lossless summary of all FHUOIs. The proposed algorithms rely on a novel weak upper bound on utility occupancy, to reduce the search space by quickly pruning itemsets with low utility occupancy. Especially, the algorithms integrate two new efficient strategies to prune non-closed FHUOI candidate branches early in the prefix search tree. Results from an in-depth experimental evaluation conducted on several benchmark real-life and synthetic quantitative datasets demonstrate that MaxCloFHUOIM and CloFHUOIM have excellent performance in terms of runtime, memory usage, and scalability. In particular, the proposed algorithms are up to two orders of magnitude faster than a baseline algorithm.

Appendices

Appendix 1: Proof of Proposition 1

• a. For any $B,C\in \left[A\right]:B\subseteq C\subseteq t,$ since ${u}_{rel}\left(B, t\right)\le {u}_{rel}\left(C, t\right)$ and $\rho \left(B\right)=\rho (C)$, then $supp\left(B\right)=supp(C)$, so $occ\left(B\right)=\frac{1}{supp\left(B\right)}{\sum }_{t\in \rho \left(B\right)}{u}_{rel}\left(B, t\right)\le \frac{1}{supp\left(C\right)}{\sum }_{t\in \rho \left(C\right)}{u}_{rel}\left(C, t\right)=occ(C)$, i.e. ${\mathcal{M}}_{\sim }(occ)$.

• b. For any $A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, we first prove that $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)\ne \mathrm{\varnothing }$. Indeed, if $A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, then $\exists {C}_{0}\stackrel{\scriptscriptstyle{\text{def}}}{=}A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)$. Otherwise, $A\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, $\exists {C}_{1}\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:$ ${C}_{1}\supset A$ and $support({C}_{1})=support(A)$. By a similar argument, if ${C}_{1}\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, then ${C}_{1}\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)$. Otherwise, ${C}_{1}\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, $\exists {C}_{2}\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:$ ${C}_{2}\supset {C}_{1}\supset A$ and $support({C}_{2})=support({C}_{1})=support(A)$, and so on. Since $\mathcal{A}$ is finite, this process must terminate, i.e. there always exists $n\in {\mathbb{N}}$: ${C}_{n}\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ with ${C}_{n}\supset \dots \supset {C}_{1}\supset A$ and $support\left({C}_{n}\right)=\dots =support({C}_{1})=support(A).$ Thus, ${C}_{n}\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)$ or $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)\ne \varnothing .$ Furthermore, assume by contradiction that there exist two different and non-nested closed FHUO itemsets $B$ and $C$ in $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)$, then for $D\stackrel{\scriptscriptstyle{\text{def}}}{=}B\bigcup C\supset B\supseteq A$, since $B,C\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)\subseteq \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and $support(C)=support(B)=support(A),$ then $\rho (C)=\rho (B)=\rho (A)$ and $\rho (D)=\rho (B)\bigcap \rho (C)=\rho (A).$ Therefore, $D\in [A],$ so $support(D)=support(A)\ge ms$ or $D\in \mathcal{F}\mathcal{I}.$ Because ${\mathcal{M}}_{\sim }(occ),$ $occ(D)\ge occ(B)\ge muo$, i.e. $\exists D\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:$ $D\supset B$ and $support(D)=support(A)$. But this contradicts the fact that $B\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}.$ Hence, $|\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\left(A\right)|=1.$ Let $C\stackrel{\scriptscriptstyle{\text{def}}}{=}closure(A),$ then by the previous argument, $C$ is the maximum itemset and has the highest utility occupancy in $[A]$.

• c. Let $\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}\stackrel{\scriptscriptstyle{\text{def}}}{=}\left\{A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I} \right| \nexists B\in \mathcal{I}\mathcal{S}:B\supset A\wedge supp\left(B\right)=supp(A)\}$. Firstly, since $A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}$ $\iff$ [$A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and $A\in \mathcal{I}\mathcal{S}:\nexists B\in \mathcal{I}\mathcal{S}:B\supset A\wedge supp\left(B\right)=supp(A)$] $\iff$ $A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I} \bigcap \mathcal{C}\mathcal{I}$, i.e. $\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}=\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I} \bigcap$ $\mathcal{C}\mathcal{I}$. Secondly, we will prove that $\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}=\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$. The first inclusion $\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}\subseteq \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ is evident. Conversely, for any $A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, i.e. $A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and $(\nexists B\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A supp\left(B\right)=supp(A))$^(*), assume by contradiction that $A\notin \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}$. Then, $\exists B\in \mathcal{I}\mathcal{S}:B\supset A: supp\left(B\right)=supp(A)$, so $\rho (B)=\rho (A)$, i.e. $A,B\in [A]$. Since ${\mathcal{M}}_{\sim }(occ)$ and $A\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, $occ\left(B\right)\ge occ\left(A\right)\ge mo$ and $supp\left(B\right)=supp(A)\ge ms$ so $B\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}.$ This contradicts ^(*). Hence, $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}$. Thus, $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}=\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{C}\mathcal{I}$. Furthermore, since $supp$ is anti-monotonic, for any $B=A+C\supset A$ with $x\in C\ne \varnothing$ and $A\subset B{\prime}=A+\{x\}\subseteq B$, we have $\rho (A)\supseteq \rho (B^{\prime})\supseteq \rho (B)$, so $supp(A)\ge supp(B^{\prime})\ge supp(B)$. Additionally, if $supp(A)=supp(B)$, then $supp(A)=supp(B^{\prime})=supp(B)$. Hence, $A$ is closed $\iff$ [$A$ has no $1-forward$ and $1-backward$]. Thus, the last assertion is proven.

Appendix 2: Proof of Theorem 1

• a. For any proper FE $B$ of $A$, $A\subset B=A\oplus E\in \mathcal{F}\mathcal{I}$ with $E\ne \varnothing$, and transaction ${t}_{i}\in \rho (B)\subseteq TS(A)\subseteq \rho (A)$, then $m\stackrel{\scriptscriptstyle{\text{def}}}{=}supp(A)\ge n\stackrel{\scriptscriptstyle{\text{def}}}{=}|TS(A)|\ge p\stackrel{\scriptscriptstyle{\text{def}}}{=}supp\left(B\right)\ge ms,$ $u\left(B,{t}_{i}\right)=u\left(A,{t}_{i}\right)+u\left(E,{t}_{i}\right)\le u\left(A,{t}_{i}\right)+u\left(rem\left(A,{t}_{i}\right)\right)={ub}_{rem}(A,$ ${t}_{i})$, so ${u}_{rel}\left(B, {t}_{i}\right)\le {ub}_{rem\_\_rel}\left(A,{t}_{i}\right)$. Since $\mathcal{X}\stackrel{\scriptscriptstyle{\text{def}}}{=}\{{wubocc}_{i}, i=1..n\}\subseteq \mathcal{Y}\stackrel{\scriptscriptstyle{\text{def}}}{=}\{{\widehat{\Phi }}_{i}, i=1..m\}$, $\mathcal{X}\downarrow$, $\mathcal{Y}\downarrow$ and $\rho (B)\subseteq TS(A)$, then the series $\{{S}_{k}\stackrel{\scriptscriptstyle{\text{def}}}{=}\frac{1}{k}{\sum }_{1\le i\le k}{wubocc}_{i},1\le k\le n\}$ is descending, so $occ\left(B\right)\stackrel{\scriptscriptstyle{\text{def}}}{=}\frac{1}{p}{\sum }_{{t}_{i}\in \rho \left(B\right)}{u}_{rel}\left(B, {t}_{i}\right)\le \frac{1}{p}{\sum }_{{t}_{i}\in \rho \left(B\right)}{ub}_{re{m}_{rel}}\left(A,{t}_{i}\right)\le {S}_{p}\le {S}_{ms}=wubocc(A)\le \frac{1}{ms}{\sum }_{1\le i\le ms}{\widehat{\Phi }}_{i}=\widehat{\Phi }(A)$.

• b. If $wubocc\left(A\right)<muo$, then $occ\left(B\right)\le wubocc\left(A\right)<muo$ for any proper FE $B$ (of $A$), which is always a low utility occupancy itemset.

Appendix 3: Proof of Theorem 2

We will prove that

$C$ is a CFHUOI ⇔ [$\left\{C\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\right|C has no 1-forward\, and\,(\nexists D\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:D\supset C\, and\, supp\left(D\right)=supp(C))$].

 + “⇒”: For any CFHUOI $C=A\oplus y$, i.e. $C\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and ($\nexists D\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:D\supset C\, and\, supp\left(D\right)=supp(C)$)^(*), assume conversely that $C$ has $1-forward$ $D=A\oplus y\oplus z$ or $(\exists D\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:D\supset C and supp\left(D\right)=supp(C))$. Then $D\supset C\, and\, supp\left(D\right)=supp(C)\ge ms$, so $D\in [C]$. Since ${M}_{\sim }(occ)$ and $C\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, $occ(D)\ge occ(C)\ge muo$, i.e. $D\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$. But this contradicts ^(*).

 + “⇐”: If [$C\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, $C$ has no $1-forward$ and $(\nexists D\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:D\supset C\, and\, supp\left(D\right)=supp(C))$] and assume conversely that $C=A\oplus y$ is not a CFHUOI, i.e. $\exists E\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:E=F\oplus y\oplus G\supset C$ with $y\succ F\supseteq A$ and $G\succ y$ (such that $F\supset A$ or $G\ne \varnothing$) and $supp\left(E\right)=supp(C)$. Since such $E\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, i.e. $E$ is not yet considered, then $F=A$ and $G\ne \varnothing$, i.e. $\exists z\in G:z\succ y$. For a special forward extension $D$ of $C$, $E\supseteq D=A\oplus y\oplus z\supset C$, we have $supp\left(E\right)=supp\left(D\right)=supp(C)$, then $D$ is a $1-forward$ of $C$, i.e. a contradiction occurs.

Appendix 4: Proof of Proposition 3

Assume conversely that $wubocc(C)<muo$ and $occ(C)\ge muo$ but $C$ has $1-forward$, i.e. $\exists D=C\oplus y\supset C:$ $supp\left(D\right)=supp(C)$ and $\rho \left(D\right)=\rho (C).$ Then, $D\in [C],$ so $muo>wubocc(C)\ge occ(D)\ge occ(C)\ge muo$ since ${\mathcal{M}}_{\sim }(occ)$. This is a contradiction. The remaining assertions are derived from Theorem 1 and 2.

Appendix 5: Proof of Proposition 4

• a. Assume conversely that $\exists A\in \mathcal{M}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ but $A\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, i.e. $\exists B\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A \bigwedge supp\left(B\right)=supp(A)$, so $A\notin \mathcal{M}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and a contradiction occurs. The remaining inclusion, $\mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, is evident.

• b. We first prove that $\mathcal{M}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \left\{A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\right| \nexists B\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A\}$. For any $A\in \mathcal{M}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, i.e. ($\nexists B\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A$)⁽⁺⁾, assume $A\notin \left\{A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\right| \nexists B\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A\}$, i.e. $\exists B\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A$, and this contradicts ⁽⁺⁾. Second, we prove the inverse inclusion. Assume that $A\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}\subseteq \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$ and ($\nexists B\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B\supset A$)⁽⁺⁺⁾, but $A\notin \mathcal{M}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, i.e. $\exists B^{\prime}\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:B^{\prime}\supset A$. Then, by Proposition 1b, $\exists B\stackrel{\scriptscriptstyle{\text{def}}}{=}closure(B^{\prime})\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}({B}^{\prime})\subseteq \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}: B\supseteq B^{\prime}\supset A$, but this contradicts ⁽⁺⁺⁾.

Appendix 6: Proof of Theorem 3

For any FE $G$ of ${B}^{\prime}$ with $F\succ y$, i.e. $G={B}{\prime}\oplus F={A}^{\prime}\oplus y\oplus F\in Br\left({B}^{\prime}\right)$. Since $A\subseteq A\mathrm{^{\prime}}\subset C$ and $D\succ y$, then for $H\stackrel{\scriptscriptstyle{\text{def}}}{=}S \bigcup F=C\oplus y\oplus (D \bigcup F)$ (when $F=\varnothing$, $H=S$ and $G={B}{\prime}$), we have $G\subset H$. Due to $supp\left(B\right)=supp(S)$ and $B\subseteq B\mathrm{^{\prime}}\subset S$, $\rho \left(B\right)=\rho \left(B^{\prime}\right)=\rho (S)$, so $\forall T\supseteq B^{\prime}$, $T\supseteq S$. Then, for any $T\supseteq G=B^{\prime}\oplus F$, we have $T\supseteq S$ and $T\supseteq F$, so $T\supseteq H$, i.e. $\rho \left(G\right)\subseteq \rho \left(H\right)$. Because the reverse inclusion is evident, we obtain $\rho \left(G\right)=\rho \left(H\right)$. Therefore, $H\in \left[G\right]$. If $occ\left(G\right)<muo$ or $supp\left(G\right)<ms$, i.e. $G\notin \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$, then $G\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$. Otherwise, if $occ\left(G\right)\ge muo$ and $supp\left(G\right)\ge ms$, then $supp\left(H\right)=supp\left(G\right)\ge ms$ and $occ\left(H\right)\ge occ\left(G\right)\ge muo$ since ${\mathcal{M}}_{\sim }\left(occ\right)$. Hence, $\exists H\in \mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:H\supset G$ and $supp\left(H\right)=supp\left(G\right)$, i.e. $G\notin \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}$. Thus, the $Br\left({B}^{\prime}\right)$ branch is $Non-CloFHUO$.

Appendix 7: Proof of Corollary 1

For $C=A\oplus y$ is being considered, since ^(*) in (1) does not hold, i.e. $\exists S\in \mathcal{C}\mathcal{F}\mathcal{H}\mathcal{U}\mathcal{O}\mathcal{I}:S\supset C and supp\left(S\right)=supp\left(C\right)$, then such $S$ must be already considered and $S=C^{\prime}\oplus y\oplus D$ with $C^{\prime}\supset A$. Thus, by Theorem 3, the $Br\left(C\right)$ branch is $Non-CloFHUO$.