FCloSM, FGenSM: two efficient algorithms for mining frequent closed and generator sequences using the local pruning strategy

Abstract

Mining frequent sequences in sequential databases are highly valuable for many real-life applications. However, in several cases, especially when databases are huge and when low minimum support thresholds are used, the cardinality of the result set can be enormous. Consequently, algorithms for discovering frequent sequences exhibit poor performance, showing an important increase in execution time, memory consumption and storage space usage. To address this issue, researchers have studied the tasks of mining frequent closed and generator sequences, as they provide several benefits when compared to the set of frequent sequences. One of the most important benefits is that the cardinalities of frequent closed and generator sequences are generally much less than the cardinality of frequent sequences. Hence, humans find it more convenient to analyze the information provided by closed and generator sequences. Moreover, it was shown that frequent closed sequences have the advantage of being lossless, and they thus preserve information about the frequency of all frequent subsequences, while generator sequences can provide higher accuracy for sequence classification tasks since they are the smallest patterns that characterize groups of sequences. Besides, frequent closed sequences can be combined with generators to produce non-redundant sequential rules and recover the complete set of frequent sequences and their frequencies. This paper proposes two novel algorithms named FCloSM and FGenSM to mine frequent closed and generator sequences efficiently. These algorithms are based on new pruning conditions called extended early elimination (3E) and early pruning techniques named EPCLO and EPGEN, designed to identify non-closed and non-generator patterns early. Based on these techniques, two local pruning strategies called LPCLO and LPGEN are proposed to eliminate non-closed and non-generator patterns more efficiently at two successive levels of the prefix search tree without performing subsequence relation checking. These theoretical results, which are the basis of FCloSM and FGenSM, are mathematically proved and are shown to be more general than those presented in previous work. Extensive experiments show that FCloSM and FGenSM are one to two orders of magnitude faster than the state-of-the-art algorithms for discovering frequent closed sequences (CloSpan, BIDE, ClaSP and CM-ClaSP) and for mining frequent generators (FEAT, FSGP and VGEN), and that FCloSM and FGenSM consume much less memory.

Appendices

Appendices

1.1 Appendix 1

To prove Proposition 1 and Theorem 1, we first consider the relation between the measures remSize and length of suffixes stated in Lemma 1.

Lemma 1

(Relation between remSize and length of suffixes).

If $\alpha \sqsubset \beta \sqsubseteq \varPsi , \textit{suf} = \textit{suf}(\varPsi , \alpha ), {suf}{'} = \textit{suf}(\varPsi , \beta )$, then

• a. $\textit{suf }\sqsupseteq \textit{suf}'$.

• b. $\textit{ suf }= \textit{suf}' \Leftrightarrow \textit{ length}(\textit{suf}) =\textit{ length}(\textit{suf}')$

  $$\begin{aligned} \Leftrightarrow \textit{remSize}(\textit{suf}) =\textit{ remSize}(\textit{suf}')\text { and }{} \textit{ lastItemOf}(\alpha ) = \textit{lastItemOf}(\beta ). \end{aligned}$$

Note that if remSize(suf) = remSize $(\textit{suf}')$, then, in the case of 1$-{\mathcal{SDB}}$s, the last condition in Lemma 1.b, lastItemOf $(\alpha ) = \textit{lastItemOf}(\beta )$, is always satisfied, but for $n-{\mathcal{SDB}}$s, this condition can be false, i.e., it is necessary to check.

Proof

Let $\delta = \textit{prefix}(\varPsi , \alpha )$ and $\delta ' =\textit{ prefix}(\varPsi ,\beta )$.

• a. It is clear that $\delta \sqsubseteq \delta '$. Thus, suf $\sqsupseteq \textit{suf}'$.

• b. The first equivalence is obviously true. Now, we prove the second equivalence.

$\bullet$ “$\Rightarrow$”: If length(suf) = length $({\textit{suf}}')$, then length $(\delta )$ = length $(\delta ')$. Hence, $\delta = \delta '$, and thus lastItemOf $(\delta )$ = lastItemOf $(\delta ')$ and size $(\delta )= \textit{size}(\delta ')$. Since remSize(suf) = size(S) - size $(\delta )$ and lastItemOf $(\alpha ) =$ lastItemOf $(\delta )$, lastItemOf $(\beta ) = \textit{lastItemOf}(\delta ')$, we obtain remSize $(\textit{suf})=\textit{ remSize}(\textit{suf}')$ and lastItemOf $(\alpha )= \textit{lastItemOf}(\beta )$.

$\bullet$ “$\Leftarrow$”: If remSize $(\textit{suf}) =\textit{ remSize}(\textit{suf}')$, then size $(\delta ) = \textit{size}(\delta ') = k$ and the $(k-$1) first events of $\delta$ and $\delta '$ are identical, so lastEventOf $(\delta ) \subseteq \textit{ lastEvent}(\delta ')$. Moreover, because $\alpha$and $\beta$ share the same last item, lastItemOf $(\alpha )= \textit{lastItemOf}(\beta )$. Thus, lastItemOf $(\delta ) = \textit{lastItemOf}(\delta ')$ and lastEventOf $(\delta )=\textit{ lastEvent}(\delta ')$. Hence, $\delta = \delta '$, $\textit{suf }= \textit{suf}'$ and length(suf) = length $({\textit{suf}}')$. $\square$

1.2 Appendix 2

To prove Theorem 1, we need to demonstrate the anti-monotonicity of the operators: $\rho$, support, PDB, SI and SE as described in Proposition 1.

Proposition 1

(Properties of SE and SI in PDBs). For two arbitrary sequences $\alpha$ and $\beta$ such that $\alpha \sqsubseteq \beta$, the following assertions hold.

• a.

  • (i). $\rho (\alpha ) \supseteq \rho (\beta )$, support $(\alpha )\geqslant \textit{ support}(\beta )$. (anti-monotonicity of the $\rho$ and support operators)

  • (ii). $\rho (\alpha )= \rho (\beta ) \Leftrightarrow \textit{ support}(\alpha )=\textit{ support}(\beta )$.

  • (iii). $\textit{support}(\alpha ) =|{\mathcal {D}}_{\alpha }|$.

• b. (Anti-monotonicity of PDB, SI, SE)

  • (i). ${\mathcal {D}}_{\alpha } \sqsupseteq {\mathcal {D}}_{\beta }$.

  • (ii). SI $({\mathcal {D}}_{\alpha }) \geqslant \textit{SI}({\mathcal {D}}_{\beta })$ and $\textit{SE}({\mathcal {D}}_{\alpha }) \geqslant \textit{ SE}({\mathcal {D}}_{\beta })$.

Proof

• a. These assertions are obviously true by the definitions of the $\rho$ and support operators.

• b.

  • (i). $\varPsi \in \rho (\alpha )\Leftrightarrow (\varPsi \in D$ and $\alpha \sqsubseteq \varPsi )$

    $\Leftrightarrow (\varPsi \in {\mathcal {D}}$ and $\varPsi = \delta \diamondsuit \; suf, \delta = \hbox {prefix}(\varPsi , \alpha )$, $\textit{suf }=\textit{ suf}(\varPsi , \alpha ))$

    $\Leftrightarrow (suf = suf(\varPsi , \alpha ) \in {\mathcal {D}}_{\alpha }$, with $\varPsi \in {\mathcal {D}}$ and $\varPsi = \delta \diamondsuit$ suf, p= prefix $(\varPsi ,\alpha ))$ or support $(\alpha ) =|\rho (\alpha )| =|{\mathcal {D}}_{\alpha }|$.

  • (ii). $\forall \varPsi \in \rho (\beta ), \textit{suf}' =$ suf $(\varPsi , \beta )\in \mathcal {D}_{\beta }$, we have $(\varPsi = \delta '\diamondsuit \textit{suf}')\wedge (\delta ' =$ prefix $(\varPsi , \beta ))$

    $\Leftrightarrow (\varPsi = \delta '\diamondsuit \textit{suf}') \wedge (\beta \sqsubseteq \delta ')\wedge (\not \exists \delta '': S = \delta ''\diamondsuit r' \wedge \beta \sqsubseteq \delta ''\sqsubset \delta ')$

    $\Rightarrow (\varPsi \in \rho (\alpha ))\wedge (\varPsi = \delta ' \diamondsuit \textit{suf}')\wedge (\alpha \sqsubseteq \delta ')$, because $\alpha \sqsubseteq \beta \sqsubseteq \delta ' \sqsubseteq \varPsi$.

  There are two possible cases:

  • If $\not \exists \delta ''$: $\varPsi = \delta ''\diamondsuit \gamma ' \wedge \alpha \sqsubseteq \delta '' \sqsubset \delta '$, then $\delta ' =\textit{ prefix}(\varPsi , \alpha )$, and thus $\textit{suf}' =\textit{suf}(\varPsi , \alpha )\in {\mathcal {D}}_{\alpha }$.

  • Otherwise, $\exists \delta '': \varPsi = \delta '' \diamondsuit \gamma ' \wedge \alpha \sqsubseteq \delta ''\sqsubset \delta '$, then $\delta ''$ is a prefix (of $\varPsi$) containing $\alpha$. We call $\gamma =\textit{ prefix}(\varPsi ,\alpha )$ the smallest prefix (of $\varPsi )$ containing $\alpha$, i.e., $\exists \textit{suf }=\textit{ suf}(\varPsi , \alpha ): (\varPsi = \gamma \diamondsuit \textit{suf }) \wedge (\alpha \sqsubseteq \gamma \sqsubseteq \delta ''\sqsubset \delta ')$, thus $\textit{suf}'\sqsubset$ suf. Therefore, suf $\in {\mathcal {D}}_{\alpha }$ and $\textit{suf}'\sqsubset \textit{ suf}$. Finally, in all cases, $\varPsi \in \rho (\alpha ), \exists$ suf $(\varPsi ,\alpha ) \in {\mathcal {D}}_{{\alpha }}$ and $\textit{suf}'\sqsubseteq \textit{ suf}$. Hence, $\mathcal {D}_{\beta } \sqsubseteq {\mathcal {D}}_{\alpha }$.

• (iii). By a. (i) and b. (ii), $\forall \varPsi \in \rho (\beta ) \subseteq \rho (\alpha ), \textit{suf}' =\textit{ suf}(\varPsi , \beta ) \in \mathcal {D}_{\beta } \textit{suf }=\textit{ suf}(\varPsi ,\alpha ) \in {\mathcal {D}}_{\alpha }$ and $\textit{suf}' \sqsubseteq$ suf, it follows that length $({\textit{suf}}')\leqslant \textit{ length}(\textit{suf})$ and remSize $({\textit{suf}}')\leqslant \textit{ remSize}(\textit{suf})$. Thus, $\textit{SI}({\mathcal {D}}_{\beta }) \leqslant \textit{ SI}({\mathcal {D}}_{\alpha })$ and SE $(\mathcal {D}_{\beta })\leqslant \textit{ SE}({\mathcal {D}}_{\alpha })$.

$\square$

1.3 Appendix 3: Proof of Theorem 1

• a. ${\mathcal {D}}_{\alpha }= \mathcal {D}_{\beta }$

  $\Leftrightarrow (\rho (\alpha )= \rho (\beta ))\wedge (\forall \varPsi \in \rho (\alpha ),\textit{ suf}=\textit{ suf}(\varPsi , \alpha ) \in {\mathcal {D}}_{\alpha }, \textit{suf}' =\textit{ suf}(\varPsi , \beta ) \in {\mathcal {D}}_{\beta },\textit{ suf }= \textit{suf}')$

  $\Rightarrow \textit{SI}({\mathcal {D}}_{\alpha })=\textit{ SI}(\mathcal {D}_{\beta })\Rightarrow \textit{ SE}({\mathcal {D}}_{\alpha })=\textit{ SE}(\mathcal {D}_{\beta })$.

Conversely, assume that SI $({\mathcal {D}}_{\alpha })=$ SI $(\mathcal {D}_{\beta })$. By a. (i) and b. (ii)–(iii) of Proposition 1 and because $\forall \varPsi \in \rho (\alpha )$, length(suf $(\varPsi , \alpha ))+1 >0$, we have that $\rho (\alpha )= \rho (\beta )$ and $\forall \varPsi \in \rho (\alpha )$, suf $=$ suf $(\varPsi , \alpha ) \in {\mathcal {D}}_{\alpha }, \textit{suf}' =\textit{ suf}(\varPsi , \beta ) \in \mathcal {D}_{\beta }, \textit{suf }\sqsupseteq \textit{suf}', \textit{ length}({\textit{suf}}')=$ length(suf), we have that suf $= \textit{suf}'$, i.e., ${\mathcal {D}}_{\alpha } = \mathcal {D}_{\beta }$ and lastItemOf $(\alpha )=$ lastItemOf $(\beta )$. Thus, SI $({\mathcal {D}}_{\alpha })=$ SI $(\mathcal {D}_{\beta })\Leftrightarrow {\mathcal {D}}_{\alpha } = \mathcal {D}_{\beta } \Rightarrow (\textit{SE}({\mathcal {D}}_{\alpha })=\textit{ SE}(\mathcal {D}_{\beta })$ and lastItemOf $(\alpha ) =\textit{lastItemOf}(\beta ))$.

Finally, assume that SE $({\mathcal {D}}_{\alpha })=\textit{ SE}(\mathcal {D}_{\beta })$. By a. (i) and b. (ii)–(iii) of Proposition 1 and $\forall \varPsi \in \rho (\alpha )$, remSize(suf $(\varPsi , \alpha )) +1 >0$, we must have $\rho (\alpha )= \rho (\beta )$, thus support $(\alpha )=\textit{ support}(\beta )$ and the first assertion b.(i) is proved. Moreover, $\forall \varPsi \in \rho (\alpha )$, suf $(\varPsi , \alpha ) \in {\mathcal {D}}_{\alpha }$, suf $(\varPsi , \beta ) \in \mathcal {D}_{\beta }$ and remSize(suf $(\varPsi , \alpha ))=\textit{ remSize}(\textit{suf}(\varPsi , \beta ))$.

Additionally, if lastItemOf $(\alpha ) =$ lastItemOf $(\beta )$, i.e., $\alpha$ and $\beta$ share the same last item, then $\forall \varPsi \in \rho (\alpha )$, length(suf $(\varPsi , \alpha ))=$ length(suf $(\varPsi , \beta ))$, so suf $(\varPsi , \alpha )=\textit{ suf}(\varPsi , \beta )$. Hence, SI $({\mathcal {D}}_{\alpha })=\textit{ SI}(\mathcal {D}_{\beta })$.

• b.

  • (ii). For any itemset A, consider two s-extensions of $\alpha$ and $\beta$ with A: $\gamma = \alpha {\diamondsuit }_{{\alpha }}A, \delta = \beta {\diamondsuit }_{{\alpha }}A$. Similarly, because of the above arguments, if SE $({\mathcal {D}}_{\alpha })=\textit{ SE}(\mathcal {D}_{\beta })$, then $\rho (\alpha )= \rho (\beta )$ and $\forall \varPsi \in \rho (\alpha )$, remSize(suf $(\varPsi ,\alpha ))=\textit{ remSize}(\textit{suf}(\varPsi , \beta ))$. Moreover, since $\gamma$ and $\delta$ share the same last event A, we obtain $\rho (\gamma )= \rho (\delta )$ and $\forall \varPsi \in \rho (\gamma )$, remSize(suf $(\varPsi , \gamma ))= \textit{remSize}(\textit{suf}(\varPsi ,\delta ))$. Therefore, $\gamma$ and $\delta$ also share the same last item ofA, so we have that length(suf $(\varPsi , \gamma ))=\textit{ length}(\textit{suf}(\varPsi ,\delta ))$. Hence, SI $({\mathcal {D}}_{\gamma }) =\textit{ SI}(\mathcal {D}_{\delta })$ and ${\mathcal {D}}_{\gamma } = \mathcal {D}_{\delta }$.

  • (iii). In addition, assume that lastEventOf $(\alpha ) =$ lastEventOf $(\beta )= A$, and for any itemset B, such that $A \prec _{alp} B$(which means that all items of A are always preceeding all items of Baccording to the total order relation $\prec _{alp})$, we consider two i-extensions of $\alpha$ and $\beta$ with B: $\gamma ' = \alpha {\diamondsuit }_{{i}}B, \delta ' = \beta {\diamondsuit }_{{i}}B$. Then, the equality of SE $({\mathcal {D}}_{\alpha })=\textit{ SE}(\mathcal {D}_{\beta })$ holds only if $\rho (\alpha )= \rho (\beta )$ and $\forall \varPsi \in \rho (\alpha )$, remSize(suf $(\varPsi , \alpha ))=\textit{ remSize}(\textit{suf}(\varPsi ,\beta ))$. Moreover, since $A \prec _{alp} B$ and lastEventOf $(\alpha )=$ lastEventOf $(\beta )= A$, then lastEventOf $(\gamma ') = \textit{lastEventOf }(\delta ') = A \cup B$, i.e., $\gamma '$ and $\delta '$ share the same last itemset $A \cup B$ and also the same last item of $A\cup B$. Therefore, $\rho (\gamma ') = \rho (\delta ')$ and $\forall \varPsi \in \rho (\gamma ')$, remSize(suf $(\varPsi$, $\gamma '))= \textit{remSize}(\textit{suf}(\varPsi , \delta '))$ and length(suf $(\varPsi , \gamma '))=\textit{ length}(\textit{suf}(\varPsi ,\delta '))$. Thus, SI $(\mathcal {D}_{\gamma '})=\textit{ SI}(\mathcal {D}_{\delta '})$ and $\mathcal {D}_{\gamma '}= \mathcal {D}_{\delta '}$.

$\square$

1.4 Appendix 4: Proof of Corollary 1

Note that for the two cases, we always have $\mathcal {D}_{\gamma } = \mathcal {D}_{\delta }$ and lastEventOf $(\gamma ) =$ lastEventOf $(\delta )$. Hence, all i-extensions and s-extensions $\gamma '$ of $\gamma$ and $\delta '$ of $\delta$with the same itemset also have the same last event, and thus, they have the same PDB, $\mathcal {D}_{\gamma '} = \mathcal {D}_{\delta '}$. The same situation then also occurs for their next descendants. $\square$

1.5 Appendix 5: Proof of Corollaries 2 and 3

Corollaries 2–3 a. (i)–(iii). These assertions are true because (1.1), path $(q)\sqsubset$ i_ new, path $(u)\sqsubset$ i_ new, and path $(r)\sqsubset$ i_ new, lastEventOf(path $(r))= \textit{lastEventOf}(\textit{i}\_\textit{ new}) =\{ q,u\}$ and (2.1).

Corollaries 2–3 b. These assertions are also true since path $(v)\sqsubset$ s_ new, (1.1) and the last events of s_ new and path(v) are identical to v and hence (2.1). $\square$