Tijl De Bie
ABSTRACT
We formalize the data mining process as a process of information exchange, defined by the following key components. The data miner's state of mind is modeled as a probability distribution, called the background distribution, which represents the uncertainty and misconceptions the data miner has about the data. This model initially incorporates any prior (possibly incorrect) beliefs a data miner has about the data. During the data mining process, properties of the data (to which we refer as patterns) are revealed to the data miner, either in batch, one by one, or even interactively. This acquisition of information in the data mining process is formalized by updates to the background distribution to account for the presence of the found patterns.
The proposed framework can be motivated using concepts from information theory and game theory. Understanding it from this perspective, it is easy to see how it can be extended to more sophisticated settings, e.g. where patterns are probabilistic functions of the data (thus allowing one to account for noise and errors in the data mining process, and allowing one to study data mining techniques based on subsampling the data). The framework then models the data mining process using concepts from information geometry, and I-projections in particular.
The framework can be used to help in designing new data mining algorithms that maximize the efficiency of the information exchange from the algorithm to the data miner.
- S.-I. Amari and H. Nagaoka. Methods of information geometry. The American Mathematical Society, Oxford University Press, 2000.Google Scholar
- E. Bae and J. Bailey. Coala: A novel approach for the extraction of an alternate clustering of high quality and high dissimilarity. In Proceedings of the International Conference on Data Mining (ICDM), 2006. Google ScholarDigital Library
- I. Csiszár and P. C. Shields. Information theory and statistics: a tutorial. now Publishers Inc., Delft, The Netherlands, 2004.Google Scholar
- T. De Bie. Explicit probabilistic models for databases and networks. Technical report, University of Bristol TR-123931, arXiv:0906.5148v1, 2009.Google Scholar
- T. De Bie. Finding interesting itemsets using a probabilistic model for binary databases. Technical report, University of Bristol TR-123930, 2009.Google Scholar
- T. De Bie. Maximum entropy models and subjective interestingness: an application to tiles in binary databases. Data Mining and Knowledge Discovery, 2010. Google ScholarDigital Library
- T. De Bie. Subjectively interesting alternative clusters. Technical report, University of Bristol TR-133090, 2011.Google Scholar
- T. De Bie, K.-N. Kontonasios, and E. Spyropoulou. A framework for mining interesting pattern sets. SIGKDD Explorations, 12(2), 2010. Google ScholarDigital Library
- S. Dzeroski. Towards a general framework for data mining. In Lecture Notes in Computer Science volume 4747, pages 259--300. springer, 2006. Google ScholarDigital Library
- C. Faloutsos and V. Megalooikonomou. On data mining, compression, and kolmogorov complexity. Data Mining and Knowledge Discovery, 15:3--20, 2007. Google ScholarDigital Library
- A. Gallo, T. De Bie, and N. Cristianini. MINI: Mining informative non-redundant itemsets. In Proceedings of Principles and Practice of Knowledge Discovery in Databases (PKDD), 2007. Google ScholarDigital Library
- A. Gallo, A. Mammone, T. De Bie, M. Turchi, and N. Cristianini. From frequent itemsets to informative patterns. Technical report, University of Bristol TR 123936, 2009.Google Scholar
- A. Gionis, H. Mannila, T. Mielikäinen, and P. Tsaparas. Assessing data mining results via swap randomization. ACM Transactions on Knowledge Discovery from Data, 1(3):14, 2007. Google ScholarDigital Library
- S. Hanhijarvi, M. Ojala, N. Vuokko, K. Puolamäki, N. Tatti, and H. Mannila. Tell me something I don't know: Randomization strategies for iterative data mining. In Proc. of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD09), pages 379--388, 2009. Google ScholarDigital Library
- P. Harremoës and F. Topsøe. Maximum entropy fundamentals. Entropy, 2001.Google Scholar
- H. Hotelling. Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24:417--441, 1933.Google ScholarCross Ref
- S. Jaroszewicz and D. A. Simovici. Interestingness of frequent itemsets using bayesian networks as background knowledge. In Proc. of the 10th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD04), pages 178--186, 2004. Google ScholarDigital Library
- S. Khuller, A. Moss, and J. Naor. The budgeted maximum coverage problem. Information Processing Letters, 70, 1999. Google ScholarDigital Library
- J. Kleinberg, C. Papadimitriou, and P. Raghavan. A microeconomic view of data mining. Data Mining and Knowledge Discovery, 2(4), 1998. Google ScholarDigital Library
- K.-N. Kontonasios and T. De Bie. An information-theoretic approach to finding informative noisy tiles in binary databases. In Proceedings of the 2010 SIAM International Conference on Data Mining, 2010.Google ScholarCross Ref
- H. Mannila. Inductive databases and condensed representations for data mining. In Proceedings of the 1997 International Symposium on Logic Programming, pages 21--30, 1997. Google ScholarDigital Library
- H. Mannila. Theoretical frameworks for data mining. SIGKDD Explorations, 2000. Google ScholarDigital Library
- H. Mannila. Randomization techniques for data mining methods. In Proc. of the 12th East European Conference on Advances in Databases and Information Systems (ADBIS08), page 1, 2008. Google ScholarDigital Library
- M. Ojala, N. Vuokko, A. Kallio, N. Haiminen, and H. Mannila. Randomization of real-valued matrices for assessing the significance of data mining results. In Proc. of the 2008 SIAM International Conference on Data Mining (SDM08), pages 494--505, 2008.Google ScholarCross Ref
- B. Padmanabhan and A. Tuzhilin. A belief-driven method for discovering unexpected patterns. In Proc. of the 4th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD98), pages 94--100, 1998.Google Scholar
- L. D. Raedt. A perspective on inductive databases. SIGKDD Explorations, 4(2):69--77, 2002. Google ScholarDigital Library
- A. Siebes, J. Vreeken, and M. van Leeuwen. Item sets that compress. In SIAM Conference on Data Mining, 2006.Google ScholarCross Ref
- A. Silberschatz and A. Tuzhilin. On subjective measures of interestingness in knowledge discovery. In Proc. of the 1st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD95), pages 275--281, 1995.Google Scholar
- N. Tatti. Maximum entropy based significance of itemsets. Knowledge and Information Systems, 17(1):57--77, 2008. Google ScholarDigital Library
- L. Valiant. A theory of the learnable. Communications of the ACM, 27, 1984. Google ScholarDigital Library
Index Terms
- An information theoretic framework for data mining
Recommendations
Analyzing the Subjective Interestingness of Association Rules
Association rules are a class of important regularities in databases. They are found to be very useful in practical applications. However, association rule mining algorithms tend to produce a huge number of rules, most of which are of no interest to the ...
From sets of good redescriptions to good sets of redescriptions
Redescription mining aims at finding pairs of queries over data variables that describe roughly the same set of observations. These redescriptions can be used to obtain different views on the same set of entities. So far, redescription mining methods ...
Uncovering the plot: detecting surprising coalitions of entities in multi-relational schemas
Many application domains such as intelligence analysis and cybersecurity require tools for the unsupervised identification of suspicious entities in multi-relational/network data. In particular, there is a need for automated semi-automated approaches to ...
Comments