Abstract
Numerous interestingness measures have been proposed in statistics and data mining to assess object relationships. This is especially important in recent studies of association or correlation pattern mining. However, it is still not clear whether there is any intrinsic relationship among many proposed measures, and which one is truly effective at gauging object relationships in large data sets. Recent studies have identified a critical property, null-(transaction) invariance, for measuring associations among events in large data sets, but many measures do not have this property. In this study, we re-examine a set of null-invariant interestingness measures and find that they can be expressed as the generalized mathematical mean, leading to a total ordering of them. Such a unified framework provides insights into the underlying philosophy of the measures and helps us understand and select the proper measure for different applications. Moreover, we propose a new measure called Imbalance Ratio to gauge the degree of skewness of a data set. We also discuss the efficient computation of interesting patterns of different null-invariant interestingness measures by proposing an algorithm, GAMiner, which complements previous studies. Experimental evaluation verifies the effectiveness of the unified framework and shows that GAMiner speeds up the state-of-the-art algorithm by an order of magnitude.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Agrawal R, Srikant R (1994, Sept) Fast algorithms for mining association rules. In: Proceedings of 1994 international conference on very large data bases (VLDB’94), Santiago, Chile, pp 487–499
Bradshaw J (2001) YAMS–Yet another measure of similarity. EuroMUG, http://www.daylight.com/meetings/emug01/bradshaw/similarity/YAMS.html
Brin S, Motwani R, Silverstein C (1997, May) Beyond market basket: generalizing association rules to correlations. In: Proceedings of 1997 ACM-SIGMOD international conference on management of data (SIGMOD’97), Tucson, AZ, pp 265–276
Brin S, Motwani R, Ullman JD, Tsur S (1997, May) Dynamic itemset counting and implication rules for market basket analysis. In: Proceedings of 1997 ACM-SIGMOD international conference on management of data (SIGMOD’97), Tucson, AZ, pp 255–264
Grahne G, Lakshmanan L, Wang X (2000, Feb) Efficient mining of constrained correlated sets. In: Proceedings of 2000 international conference on data engineering (ICDE’00), San Diego, CA, pp 512–521
Han J, Cheng H, Xin D, Yan X (2007) Frequent pattern mining: current status and future directions. Data Min Knowl Discov 15: 55–86
Han J, Pei J, Yin Y (2000, May) Mining frequent patterns without candidate generation. In: Proceedings of 2000 ACM-SIGMOD international conference on management of data (SIGMOD’00), Dallas, TX, pp 1–12
He B, Chang KC-C, Han J (2004, Aug) Discovering complex matchings across web query interfaces: a correlation mining approach. In: Proceedings of 2004 ACM SIGKDD international conference on knowledge discovery in databases (KDD’04), Seattle, WA, pp 148–157
Hilderman RJ, Hamilton HJ (2001) Knowledge discovery and measures of interest. Kluwer Academic, Dordrecht
Kachigan S (1991) Multivariate statistical analysis: a conceptual introduction. Radius Press, New York
Kulczynski S (1927) Die pflanzenassoziationen der pieninen. In: Bulletin International de l’Académie Polonaise des Sciences et des Lettres, pp 57–203
Lee Y-K, Kim W-Y, Cai YD, Han J (2003, Nov) CoMine: efficient mining of correlated patterns. In: Proceedings of 2003 international conference on data mining (ICDM’03), Melbourne, FL, pp 581–584
Omiecinski E (2003) Alternative interest measures for mining associations. IEEE Trans Knowl Data Eng 15: 57–69
Polya G, Harold HG, Littlewood JE (1988) Inequalities. Cambridge University Press, Cambridge
Savasere A, Omiecinski E, Navathe S (1998, Feb) Mining for strong negative associations in a large database of customer transactions. In: Proceedings of 1998 international conference on data engineering (ICDE’98), Orlando, FL, pp 494–502
Tan P-N, Kumar V, Srivastava J (2002, July) Selecting the right interestingness measure for association patterns. In: Proceedings of 2002 ACM SIGKDD international conference on knowledge discovery in databases (KDD’02), Edmonton, Canada, pp 32–41
Wu T, Chen Y, Han J (2007, Sept) Association mining in large databases: a re-examination of its measures. In: Proceedings of 2007 international conference on principles and practice of knowledge discovery in databases (PKDD’07), Warsaw, Poland, pp 621–628
Xiong H, Shekhar S, Tan P-N, Kumar V (2004, Aug) Exploiting a support-based upper bound of Pearson’s correlation coefficient for efficiently identifying strongly correlated pairs. In: Proceedings of 2004 ACM SIGKDD international conference on knowledge discovery in databases (KDD’04), Seattle, WA, pp 334–343
Author information
Authors and Affiliations
Corresponding author
Additional information
Responsible editor: M.J. Zaki.
The work was supported in part by the U.S. National Science Foundation NSF IIS-08-42769, NSF IIS-09-05215, NSF DMS-05-03981, and NSF DMS-08-06175. Any opinions, findings, and conclusions or recommendations expressed in this paper are those of the authors and do not necessarily reflect the views of the funding agencies. This paper is a major-value added extension of the paper: Tianyi Wu, Yuguo Chen and Jiawei Han, Association Mining in Large Databases: A Re-Examination of Its Measures”, Proc. 2007 Int. Conf. on Principles and Practice of Knowledge Discovery in Databases (PKDD’07), Warsaw, Poland, Sept. 2007, pp. 621–628.
Rights and permissions
About this article
Cite this article
Wu, T., Chen, Y. & Han, J. Re-examination of interestingness measures in pattern mining: a unified framework. Data Min Knowl Disc 21, 371–397 (2010). https://doi.org/10.1007/s10618-009-0161-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10618-009-0161-2