Scaling up cosine interesting pattern discovery: A depth-first method

Abstract

This paper presents an efficient algorithm called ${\text{CosMiner}}_t$ for interesting pattern discovery. The widely used cosine similarity, found to possess the null-invariance property and the anti-cross-support-pattern property, is adopted as the interestingness measure in ${\text{CosMiner}}_t$. ${\text{CosMiner}}_t$ is generally an FP-growth-like depth-first traversal algorithm that rests on an important property of the cosine similarity: the conditional anti-monotone property (CAMP). The combined use of CAMP and the depth-first support-ascending traversal strategy enables the pre-pruning of uninteresting patterns during the mining process of ${\text{CosMiner}}_t$. Extensive experiments demonstrate the high efficiency of ${\text{CosMiner}}_t$ in interesting pattern discovery, in comparison to the breath-first strategy and the post-evaluation strategy. In particular, ${\text{CosMiner}}_t$ shows its capability in suppressing the generation of cross-support patterns and discovering rare but truly interesting patterns. Finally, an interesting case of landmark recognition is presented to illustrate the value of cosine interesting patterns found by ${\text{CosMiner}}_t$ in real-world applications.

Introduction

Since the introduction of frequent patterns in the early nineties of the last century [1], association analysis has become one of the core problems in the data mining and database communities [2]. Given a large set of items (objects) and the observation data about co-occurring items, association analysis is concerned with the identification of strongly correlated subsets of items [27]. It plays an important role in many application domains such as market-basket analysis [3], [8], recommender systems [22], telecommunication network alarm analysis [26], image processing [25], climate studies [32], public health [10], and bio-informatics [38].

It was not until recently, researchers found that the classic support–confidence framework of association analysis has some intrinsic defects. For instance, the well-known “coffee–tea” example reveals that the confidence as an interestingness measure for association rules may not disclose truly interesting relationships [13], [14]. Also, this framework often generates too many frequent patterns and even more association rules, from which to find useful patterns is itself a great challenge. Meanwhile, the support-based pruning strategy is not effective for data sets with skewed support distributions [40]; that is, if the minimum support threshold is relatively low, we may extract too many spurious patterns involving items with substantially different support levels, such as {earrings, milk} in which the support of earrings is much lower than that of milk. On the contrary, if the threshold is high, we may miss many interesting but relatively infrequent patterns [16], such as {earrings, gold ring, bracelet} that contains rare but truly valuable items.

To overcome these problems, many interestingness measures have been proposed to fine truly interesting patterns [12]. Patterns with a measure value above some given threshold are called interesting patterns with respect to that measure. Among the proposed measures, the cosine similarity gains particular interests. Indeed, cosine similarity has various merits for association analysis. First, it is one of the few interestingness measures that hold symmetry, null-invariance and anti-cross-support-pattern properties simultaneously (details will be given in Section 2.2). Many well-known measures, such as confidence and lift, cannot achieve this. Second, cosine similarity has been widely used as proximity measure in text mining [41], information retrieval [17], image processing [11], and bio-informatics [20], to avoid the “curse of dimensionality” [4] – a problem we also face in association analysis. Finally, it is very simple and has physical meaning, i.e., it measures the angle of two vectors in the feature space. As a result, in this paper, we select the cosine similarity as the interestingness measure to define interesting patterns.

Mining cosine interesting patterns is by no means a trivial task. First of all, the traditional definition of cosine similarity is only for two vectors, which must be extended to the case of multi-itemsets. More importantly, the cosine similarity may not hold the anti-monotone property – the key for the success of frequent pattern mining using the support measure – and therefore may not be able to reduce the search space of interesting patterns. One may argue that a POST-evaluation scheme can already work, by first generating frequent patterns, and then identifying interesting patterns from frequent patterns using a cosine threshold. While being simple, the post-evaluation scheme has the same problem as the traditional frequent pattern mining algorithms; that is, rare but interesting patterns will be lost if a higher support threshold is set, but too many patterns will be generated for a low threshold instead.

In light of these, in this paper, we attempt to design an efficient algorithm to mine cosine interesting patterns from large-scale databases. To this end, we first expand the definition of cosine similarity to the scope of multi-itemsets, and highlight some valuable properties of the cosine similarity, such as the null-invariance property and the anti-cross-support-pattern property. We then point out that the cosine similarity does not hold the anti-monotone property, and propose a novel conditional anti-monotone property (CAMP) as an alternative. We argue that a depth-first itemset traversal strategy is more appropriate than a breath-first strategy when mining cosine interesting pattern using CAMP, based on which an algorithm named ${\text{CosMiner}}_t$ is proposed. ${\text{CosMiner}}_t$ is an FP-growth-like algorithm that uses the cosine as well as support thresholds to prune uninteresting patterns, although the mining of single-path tree is different from the FP-growth algorithm [15]. Extensive experiments on various real-world data sets demonstrate the superiority of ${\text{CosMiner}}_t$ to the Apriori-like method (${\text{CosMiner}}_a$) and the post-evaluation method (POST), in terms of efficiency. We also verify the abilities of ${\text{CosMiner}}_t$ in suppressing cross-support patterns and mining the rare but interesting patterns. Finally, we apply ${\text{CosMiner}}_t$ as a noise-removal tool to a real-world landmark recognition case, based on which the performance of image clustering is improved substantially. This, in turn, demonstrates the value of cosine interesting patterns mined by ${\text{CosMiner}}_t$.

The remainder of this paper is organized as follows. In Section 2, we define and explore the cosine similarity from a pattern mining perspective. In Section 3, we propose the conditional anti-monotone property and suggest the combined use of a depth-first traversal strategy. The ${\text{CosMiner}}_t$ algorithm is proposed in Section 4. Section 5 shows the experimental results, followed by a case study on landmark recognition in Section 6. We finally present the related work and conclude our work in Section 7 and Section 8, respectively.