A user parameter-free approach for mining robust sequential classification rules

Abstract

Sequential data are generated in many domains of science and technology. Although many studies have been carried out for sequence classification in the past decade, the problem is still a challenge, particularly for pattern-based methods. We identify two important issues related to pattern-based sequence classification, which motivate the present work: the curse of parameter tuning and the instability of common interestingness measures. To alleviate these issues, we suggest a new approach and framework for mining sequential rule patterns for classification purpose. We introduce a space of rule pattern models and a prior distribution defined on this model space. From this model space, we define a Bayesian criterion for evaluating the interest of sequential patterns. We also develop a user parameter-free algorithm to efficiently mine sequential patterns from the model space. Extensive experiments show that (i) the new criterion identifies interesting and robust patterns, (ii) the direct use of the mined rules as new features in a classification process demonstrates higher inductive performance than the state-of-the-art sequential pattern-based classifiers.

Appendix: Proof of Theorem 1 and  2

Here, we will present a complete Proof of Theorem 1, while theorem 2 can be proven in the same way.

Proof

Given the cost of the null model in Eq. , the prior terms are neglected when the number of sequences n is very high. Therefore, when $n \rightarrow \infty$ the cost of the null model is written as:

$$\begin{aligned} \mathrm{cost}(\pi _{\emptyset })= & {} log (n!)- \sum _{i=1}^{j} log (n_{c_i}!)\\ \end{aligned}$$

Using the approximation $log(n!) = n(log(n)-1)+O(log(n))$ based on Stirlings formula, the cost of the null model can be written as:

$$\begin{aligned} \mathrm{cost}(\pi _{\emptyset })= & {} n(log(n)-1)- \sum _{i=1}^{j} n_{c_i}(log (n_{c_i})-1)+O(log(n))\nonumber \\= & {} nlog(n)-n - \sum _{i=1}^{j} n_{c_i}log (n_{c_i})+ \sum _{i=1}^{j} n_{c_i}+O(log(n))\\ \end{aligned}$$

Notice that $n=\sum _{i=1}^{j} n_{c_i}$. Therefore, we have:

$$\begin{aligned} \mathrm{cost}(\pi _{\emptyset })= & {} \sum _{i=1}^{j} n_{c_i}log(n)-n - \sum _{i=1}^{j} n_{c_i}log (n_{c_i})+ n+O(log(n))\\= & {} -\sum _{i=1}^{j} n_{c_i} \bigg ( log(n_{c_i})-log(n) \bigg )+O(log(n))\\= & {} n \times \bigg (-\sum _{i=1}^{j}\frac{n_{c_i}}{n}log( \frac{n_{c_i}}{n})\bigg )+O(log(n))\\= & {} n \times \bigg (-\sum \limits _{i=1}^{j} p(c_i)log(p(c_i))\bigg )+O(log(n))\\= & {} n \times H(y) +O(log(n)) \end{aligned}$$