Size matters: choosing the most informative set of window lengths for mining patterns in event sequences

Abstract

In order to find patterns in data, it is often necessary to aggregate or summarise data at a higher level of granularity. Selecting the appropriate granularity is a challenging task and often no principled solutions exist. This problem is particularly relevant in analysis of data with sequential structure. We consider this problem for a specific type of data, namely event sequences. We introduce the problem of finding the best set of window lengths for analysis of event sequences for algorithms with real-valued output. We present suitable criteria for choosing one or multiple window lengths and show that these naturally translate into a computational optimisation problem. We show that the problem is NP-hard in general, but that it can be approximated efficiently and even analytically in certain cases. We give examples of tasks that demonstrate the applicability of the problem and present extensive experiments on both synthetic data and real data from several domains. We find that the method works well in practice, and that the optimal sets of window lengths themselves can provide new insight into the data.

Appendix: Proof of Theorem 1

Preliminaries Let $(X_1,\ldots ,X_n)$ be a sequence of Bernoulli random variables with common parameter $p$, i.e., $X_{i} \in \left\{ {0,1}\right\} , {{\mathrm{Pr}}}\left( {\left\{ {X_{i} = 1}\right\} }\right) = p$, for all $i \in \left\{ {1,\ldots ,n}\right\}$. The random variables could, for example, denote the occurrences of an event. Similar to the notation for event sequences, we use $X_{i,\omega }$ to denote the subsequence of length $\omega$ starting at position $i$, $(X_i,\ldots ,X_{i+\omega -1})$. Let the statistic $f$ be the relative frequency of ones:

$$\begin{aligned} f(X_{i,\omega })=\frac{1}{\omega } \sum _{j=i}^{i+\omega -1}{X_j}. \end{aligned}$$

(5)

The selection of an optimal set of window lengths is based on the squared error between predictions made using those window lengths (Problem 1). Under the constraint of using a $k$-partition nearest neighbour regressor, the predictions correspond to the value of the nearest window length (Sect. 4.1). Thus, to select the optimal window lengths, we have to compute the distance (squared error) between all pairs of window lengths. We find that the distance between window lengths is as follows.

Theorem 1

For the statistic and generative process described above, the expected distance between two window lengths $\gamma$ and $\omega$, with $\gamma < \omega$, is

$$\begin{aligned} {{\mathrm{E}}}\left[ {d(\omega ,\gamma )}\right] = \frac{\omega -\gamma }{\omega \gamma } p (1-p). \end{aligned}$$

Proof

The expected distance between two window lengths $\gamma$ and $\omega$ is

$$\begin{aligned} {{\mathrm{E}}}\left[ {d(\omega ,\gamma )}\right] = {{\mathrm{E}}}\left[ {\frac{1}{n^{*}} \sum _{i=1}^{n^{*}} \left( f(X_{i,\gamma })-f(X_{i,\omega })\right) ^{2}}\right] . \end{aligned}$$

Since $X_1,\ldots ,X_n$ are i.i.d. random variables, this simplifies to

$$\begin{aligned} {{\mathrm{E}}}\left[ {d(\omega ,\gamma )}\right] = {{\mathrm{E}}}\left[ {\left( f(X_{1,\gamma })-f(X_{1,\omega })\right) ^{2}}\right] . \end{aligned}$$

Assuming without loss of generality that $\gamma < \omega$, we find that

$$\begin{aligned} f(X_{1,\omega })&= \frac{1}{\omega } \sum _{j=1}^{\omega } X_{j}\\&= \frac{1}{\omega }\sum _{j=1}^{\gamma } X_{j}+\frac{1}{\omega } \sum _{j=1+\gamma }^{\omega } X_{j}\\&= \frac{\gamma }{\omega } f(X_{1,\gamma }) + \frac{\omega - \gamma }{\omega } f(X_{1+\gamma ,\omega -\gamma }). \end{aligned}$$

Thus we can rewrite the expected distance as

$$\begin{aligned}&{{\mathrm{E}}}\left[ {d(\omega ,\gamma )}\right] \\&\quad = {{\mathrm{E}}}\left[ {\left( f(X_{1,\gamma }) - \frac{\gamma }{\omega } f(X_{1,\gamma }) - \frac{\omega -\gamma }{\omega } f(X_{1+\gamma ,\omega -\gamma })\right) ^{2}}\right] \\&\quad = {{\mathrm{E}}}\left[ {\left( \frac{\omega -\gamma }{\omega }\right) ^{2}\left( f(X_{1, \gamma }) - f(X_{1+\gamma ,\omega -\gamma })\right) ^{2}}\right] \\&\quad = \left( \frac{\omega -\gamma }{\omega }\right) ^{2} {{\mathrm{E}}}\left[ {\left( f(X_{1, \gamma }) - f(X_{1+\gamma ,\omega -\gamma })\right) ^{2}}\right] \\&\quad = \left( \frac{\omega -\gamma }{\omega }\right) ^{2} {{\mathrm{E}}}\left[ {f(X_{1, \gamma })^{2}}\right] + {{\mathrm{E}}}\left[ {f(X_{1+\gamma ,\omega -\gamma })^{2}}\right] - 2 {{\mathrm{E}}}\left[ {f(X_{1, \gamma }) f(X_{1+\gamma ,\omega -\gamma })}\right] . \end{aligned}$$

These three expectations are

$$\begin{aligned} {{\mathrm{E}}}\left[ {f(X_{1,\gamma })^{2}}\right]&= \frac{p(1-p)}{\gamma }+p^2,\\ {{\mathrm{E}}}\left[ {f(X_{1+\gamma ,\omega -\gamma })^{2}}\right]&= \frac{p(1-p)}{\omega -\gamma }+p^2, \text { and}\\ {{\mathrm{E}}}\left[ {f(X_{1,\gamma }) f(X_{1+\gamma ,\omega -\gamma })}\right]&= p^2. \end{aligned}$$

For brevity, we skip the derivation for these three expectations. They can be derived, for example, using the fact that the variance of a binomial distribution is ${{\mathrm{Var}}}\left[ {Bin(n,p)}\right] = {{\mathrm{E}}}\left[ {Bin(n,p)^2}\right] -{{\mathrm{E}}}\left[ {Bin(n,p)}\right] ^2 = np(1-p)$, and its expectation is ${{\mathrm{E}}}\left[ {Bin(n,p)}\right] = np$.

By writing out the expected distance we find that

$$\begin{aligned} {{\mathrm{E}}}\left[ {d(\omega ,\gamma )}\right]&= \left( \frac{\omega -\gamma }{\omega }\right) ^{2} \frac{p(1-p)}{\gamma }+p^2 + \frac{p(1-p)}{\omega -\gamma }+p^2 - 2 p^2\\&= \left( \frac{\omega -\gamma }{\omega }\right) ^{2} \left( \frac{1}{\gamma }+ \frac{1}{\omega -\gamma }\right) p(1-p)\\&= \frac{(\omega -\gamma )^2}{\omega ^2} \frac{\omega -\gamma +\gamma }{\gamma (\omega -\gamma )} p(1-p)\\&= \frac{\omega -\gamma }{\omega \gamma } p(1-p). \end{aligned}$$