Frequent high minimum average utility sequence mining with constraints in dynamic databases using efficient pruning strategies

Abstract

High utility sequence mining is a popular data mining task, which aims at finding sequences having a high utility (importance) in a quantitative sequence database. Though it has several applications, state-of-the-art algorithms have one or more of the following limitations: (1) they rely on a utility function that tends to be biased toward finding long patterns, (2) some algorithms do take pattern length into account using an average-utility function but they adopt an optimistic perspective that can be risky or misleading for some applications, (3) they do not let the user specify additional constraints on patterns to be found. To address these three limitations, this paper defines a novel task of mining frequent high minimum average-utility sequences (FHAUS) with constraints in a quantitative sequence database. This task has the following benefits. First, it uses the average-utility au function based on the minimum utility, which takes the length of a pattern into account to calculate its utility. This helps finding short patterns missed by traditional algorithms and it is based on more safe pessimistic utility calculations. Second, the user can specify a set of monotonic and anti-monotonic constraints C on patterns to filter irrelevant patterns and improve the performance of the mining process. To efficiently find all FHAUSs with constraints, this paper first proposes some novel upper bounds (UBs) and weak upper bounds (WUBs) on the average-utility, which satisfy downward-closure (DC) properties or DC-like properties. Then, to effectively reduce the search space, the paper designs novel width pruning, depth pruning, reducing, and tightening strategies based on the proposed bounds. These proposed novel theoretical results are integrated into an algorithm named C-FHAUSPM (Constrained Frequent High minimum Average-Utility Sequential Pattern Mining) for efficiently discovering all FHAUSs with constraints. Results from extensive experiments on both real-life and synthetic quantitative sequence databases show that C-FHAUSPM is highly efficient in terms of runtime and memory usage.

Appendices

Appendices

1.1 Appendix 1 (Proof of Theorem 1)

To prove Theorem 1, the following lemma is introduced.

Lemma 1 (Properties of taub)

For each ${\varPsi}^{\prime}\in \mathcal{D}\hbox{'}$, consider any sequence α and its forward extension β in Ψ^', and m  =  length(β)  ≥  l  =  length(α).

• a. The sequence $\left\{\frac{1}{k}{S}_k^{\alpha },1\le k\le n\right\}$ is decreasing.

• b. $\mathcal{T}\left(\beta, {\varPsi}^{\prime}\right)\subseteq \mathcal{T}\left(\alpha, {\varPsi}^{\prime}\right)$ and $\frac{1}{m}{S}_m^{\beta}\le \frac{1}{m}{S}_m^{\alpha }$.

• c. AM_aub(α,  Ψ^') and taub(α,  Ψ^') are gradually tighter UBs on au(α,  Ψ^') in each Ψ^', i.e., AM_aub(α,  Ψ^')  ≥  taub(α,  Ψ^')  ≥  au(α,  Ψ^'),  ∀  α  ⊆  Ψ^'.

• d. $\mathcal{AMF}\left( taub,{\varPsi}^{\prime}\right)$, or taub(α,  Ψ^') is anti-monotone w.r.t forward extensions in each Ψ^', i.e., taub(β,  Ψ^')  ≤  taub(α,  Ψ^'), for any forward extension β of α in Ψ^'.

Proof.

The two assertions a and d are proven similarly to Lemma 1 in.

• b. For any forward extension β of α in Ψ^', Ψ^'  ⊇  β  =  α  ⋄  δ  ⊇  α, since ${\alpha}_{first}^{\prime}\subseteq {\beta}_{first}^{\prime}\subseteq {\varPsi}^{\prime }$, $lastItem\left({\beta}_{first}^{\prime}\right)$ does not precede $lastItem\left({\alpha}_{first}^{\prime}\right)$ (in Ψ^'). Thus, $\mathcal{V}\left(\beta, {\varPsi}^{\prime}\right)\subseteq \mathcal{V}\left(\alpha, {\varPsi}^{\prime}\right)$ and $\mathcal{T}\left(\beta, {\varPsi}^{\prime}\right)\subseteq \mathcal{T}\left(\alpha, {\varPsi}^{\prime}\right)$, so $\frac{1}{m}{S}_m^{\beta}\le \frac{1}{m}{S}_m^{\alpha }$.

• c. For any ${\alpha}^{\prime}\in \mathcal{O}\left(\alpha, {\varPsi}^{\prime}\right),$ we always have l  =  length(α'), $au\left({\alpha}_{first}^{\prime}\right)\le \frac{1}{l}{S}_l^{\alpha }= taub\left(\alpha, {\varPsi}^{\prime}\right)$, so $au\left(\alpha, {\varPsi}^{\prime}\right)=\min \left\{ au\left({\alpha}^{\prime}\right)|{\alpha}^{\prime}\in \mathcal{O}\left(\alpha, {\varPsi}^{\prime}\right)\right\}\le au\left({\alpha}_{first}^{\prime}\right)\le taub\left(\alpha, {\varPsi}^{\prime}\right)$. Moreover, since ${S}_l^{\alpha}\le \min \left\{u\left({\varPsi}^{\prime}\right),l\times \max \right\{q\mid$   (a,  q)  ∈  Ψ^'}},  ∀  α  ⊆  Ψ^', it follows that taub(α,  Ψ^')  ≤  AM_aub(α,  Ψ^').

Proof of Theorem 1.

• a. (i). For any super-sequence β of α, β  ⊇  α, since ρ(β)  ⊆  ρ(α) and m  =  length(β)  ≥  l  =  length(α), then $\frac{1}{m}u\left({\varPsi}^{\prime}\right)\le \frac{1}{l}u\left({\varPsi}^{\prime}\right)$ and AM_aub(β)  ≤  AM_aub(α), i.e., $\mathcal{AM}\left( AM\_ aub\right)$.

• (ii). By Lemma 1.d, for any forward extension β of α in Ψ^', since ρ(β)  ⊆  ρ(α) and taub(β,  Ψ^')  ≤  taub(α,  Ψ^'), then$t\_ aub\left(\beta \right)={\sum}_{\varPsi^{\prime}\in \rho \left(\beta \right)} taub\left(\beta, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\beta, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\alpha, {\varPsi}^{\prime}\right)=t\_ aub\left(\alpha \right).$ Thus, $\mathcal{AMF}\Big(t\_ aub$).

• (iii). Without loss of the generality, we only consider any forward extension β  =  α  ⋄  z of α  =  ε  ⋄  y. By Lemma 1.d, $l\_ aub\left(\beta \right)={\sum}_{\varPsi^{\prime}\in \rho \left(\beta \right)} taub\left(\alpha, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\varepsilon, {\varPsi}^{\prime}\right)=l\_ aub\left(\alpha \right)$. Similarly, for any backward extension β  =  ε  ⋄  δ  ⋄  y of α  =  ε  ⋄  y, since ε  ⋄  δ is a forward extension ε, we also have $l\_ aub\left(\beta \right)={\sum}_{\varPsi^{\prime}\in \rho \left(\beta \right)} taub\left(\varepsilon \diamond \delta, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\varepsilon, {\varPsi}^{\prime}\right)=l\_ aub\left(\alpha \right)$. Hence, $\mathcal{AMB}i\left(l\_ aub\right)$.

• b. + First, we prove that t_aub and l_aub are UBs on au, and t_aub is tighter than l_aub. For any β  =  α  ⋄  δ  ⊃  α and Ψ^'  ∈  ρ(β)  ⊆  ρ(α), by Lemma 1.c and d, we have $au\left(\alpha \right)={\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} au\left(\alpha, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\alpha, {\varPsi}^{\prime}\right)=t\_ aub\left(\alpha \right)$, i.e., t_aub is an UB on au. Moreover, $t\_ aub\left(\alpha \diamond y\right)\overset{\mathrm{def}}{=}{\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \diamond y\right)} taub\left(\alpha \diamond y,{\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \diamond y\right)} taub\left(\alpha, {\varPsi}^{\prime}\right)=l\_ aub\left(\alpha \diamond y\right)$, so t_aub is tighter than l_aub. However, as shown in Example 3, l_aub and AM_aub are incomparable.

+ Next, we prove that t_waub is a WUB on au, and it is tighter than t_aub. By Lemma 1.a, $twaub\left(\alpha, {\varPsi}^{\prime}\right)\le topwaub\left(\alpha, {\varPsi}^{\prime}\right)=\frac{1}{l+1}{S}_{l+1}^{\alpha}\le \frac{1}{l}{S}_l^{\alpha }= taub\left(\alpha, {\varPsi}^{\prime}\right)$. Besides, since PE(α)  ⊆  ρ(α), then $t\_ waub\left(\alpha \right)={\sum}_{\varPsi^{\prime}\in SC\left(\alpha \right)} twaub\left(\alpha, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in \rho \left(\alpha \right)} taub\left(\alpha, {\varPsi}^{\prime}\right)=t\_ aub\left(\alpha \right)$. By Theorem 1.a, t_waub is tighter than t_aub.

Moreover, for any proper (forward) extension β  =  α  ⋄  δ of α with δ≠ <> and any m: such that $l<m= length\left(\beta \right)\le n=\mid \mathcal{T}\left(\alpha, {\varPsi}^{\prime}\right)\mid$, consider Ψ^'  ∈  ρ(β)  ⊆  PE(α)  ⊆  ρ(α).

(i). Since $\mathcal{T}\left(\beta, {\varPsi}^{\prime}\right)\subseteq \mathcal{T}\left(\alpha, {\varPsi}^{\prime}\right)$ by Lemma 1.b-c, we have $au\left(\beta, {\varPsi}^{\prime}\right)\le taub\left(\beta, {\varPsi}^{\prime}\right)=\frac{1}{m}{S}_m^{\beta}\le \frac{1}{m}{S}_m^{\alpha}\le \frac{1}{l+1}{S}_{l+1}^{\alpha }= topwaub\left(\alpha, {\varPsi}^{\prime}\right)$.

(ii). On the other hand, for any ${\beta}^{\prime}\in \mathcal{O}\left(\beta, {\varPsi}^{\prime}\right)$, there exists ${\alpha}^{\prime}\in \mathcal{O}\left(\alpha, {\varPsi}^{\prime}\right):{\beta}^{\prime }={\alpha}^{\prime}\diamond {\delta}^{\prime }$, length(rem(α^',  Ψ^'))  >  0, 1  ≤  k  =  length(δ^')  ≤  length(rem(α^',  Ψ^')) and length(β^')  =  m  =  l  +  k, $au\left({\beta}^{\prime}\right)=\frac{1}{m}\left(u\left({\alpha}^{\prime}\right)+{\sum}_{\left({a}_i,{q}_i\right)\in {\delta}^{\prime }}{q}_i\right)\le \frac{l.{au}^{\prime }+k. Mu}{l+k}=g(k)$. Since ${g}^{\prime }(k)=\frac{l.\left( Mu-{au}^{\prime}\right)}{{\left(l+k\right)}^2}$ is in the [1,  length(rem(α^',  Ψ^'))] interval, if Mu  >  au^', the function g(m) increases, so g(k)  ≤  g(length(rem(α^',  Ψ^')))  =  remwaub^'(α^',  Ψ^'). Otherwise, g(k) decreases, so g(k)  ≤  g(1)  =  remwaub^'(α^',  Ψ^'). Thus, we always have au(β^')  ≤  remwaub^'(α^',  Ψ^').Hence, au(β,  Ψ^') $=\min \left\{ au\left({\beta}^{\prime}\right)|{\beta}^{\prime}\in \mathcal{O}\left(\beta, {\varPsi}^{\prime}\right)\right\}$ ≤ max{ ${remwaub}^{\prime}\left({\alpha}^{\prime },{\varPsi}^{\prime}\right)\mid {\alpha}^{\prime}\in \mathcal{O}\left(\alpha, {\varPsi}^{\prime}\right)\wedge length\left( rem\left({\alpha}^{\prime },{\varPsi}^{\prime}\right)\right)>0\Big\}=\max \left\{{remwaub}^{\prime}\left({\alpha}^{\prime },{\varPsi}^{\prime}\right)|{\alpha}^{\prime}\in \mathcal{O}\left(\alpha, {\varPsi}^{\prime}\right)\wedge rem\left({\alpha}^{\prime },{\varPsi}^{\prime}\right)\ne <>\right\}$ =remwaub(α, Ψ^').

From (i) and (ii), we have au(β,  Ψ^')  ≤    min  {topwau b(α,  Ψ^'),  remwaub(α,  Ψ^')}  =  twaub(α,  Ψ^'). Since ρ(β)  ⊆  PE(α), we obtain $au\left(\beta \right)={\sum}_{\varPsi^{\prime}\in \rho \left(\beta \right)} au\left(\beta, {\varPsi}^{\prime}\right)\le {\sum}_{\varPsi^{\prime}\in PE\left(\alpha \right)} twaub\left(\alpha, {\varPsi}^{\prime}\right)=t\_ waub\left(\alpha \right)$. Thus, t_waub is a WUB on au.

1.2 Appendix 2 (Proof of Theorem 2)

• a. If (t_aub(α)  <  mu), then by Theorem 1.a.(ii), ∀β  =  α  ⋄  δ  ⊇  α, ρ(β)  ⊆  ρ(α), au(β)  ≤  t_aub(β)  ≤  t_aub(α)  <  mu, i.e., we can deeply prune the branch(α).

• b. If (t_waub(α)  <  mu), then for any proper forward extension β  =  α  ⋄  δ(⊃α) of α, we have au(β)  ≤  t_waub(α)  <  mu. Hence, propBranch(α) can be pruned.

• c. If WidthPC_{l_aub}(α), for example l_aub(α)  <  mu, we can also prune branch(α) because $\mathcal{AMB}i\left(l\_ aub\right)$ yields $\mathcal{AMF}\left(l\_ aub\right)$. Besides, since $\mathcal{AMB}i\left(l\_ aub\right)$, the remaining assertions are also true. Indeed, for example, for any y  ∈  I(α⋄_ix), then y  ≻  x, l_aub(α⋄_ix⋄_iy)  ≥  mu. Thus, l_aub(α⋄_iy)  ≥  l_aub(α⋄_i x⋄_iy)  ≥  mu, so y  ∈  I(α), i.e., I(α⋄_ix)  ⊆  I(α). The remaining assertions are similarly proven.