Abstract
Information-theoretic viewpoint at the data-based model construction is anchored on the assumption that both source data and a constructed model comprises certain information. Not having another source of information than source data, the process of model construction can be viewed at as the transformation of information representation. The combination of this basic idea with the Minimum Description Length principle brings a new restriction on the process of model learning: avoid models containing more information than source data, because these models must comprise an additional undesirable information. In the paper, the idea is explained and illustrated on the data-based construction of multidimensional probabilistic compositional models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Csiszár, I.: I-divergence Geometry of Probability Distributions and Minimization problems. Ann. probab. 3, 146–158 (1975)
Grünwald, P.: A Tutorial Introduction to the Minimum Description Length Principle, p. 80 (2004). [cit. 2014–07-15] http://eprints.pascal-network.org/archive/00000164/01/mdlintro.pdf
Hansen, M.H., Bin, Y.U.: Minimum Description Length Model Selection Criteria for Generalized Linear Models, p. 20. http://www.stat.ucla.edu/cocteau/papers/pdf/glmdl.pdf
Huffman, D.A.: A Method for the Construction of Minimum-Redundancy Codes. Proceedings of the I.R.E., 1098–1102 (1952)
Jensen, F.V.: Bayesian Networks and Decision Graphs. IEEE Computer Society Press, New York (2001)
Jiroušek, R.: Foundations of compositional model theory. Int. J. General Systems 40(6), 623–678 (2011)
Jiroušek, R., Shenoy, P.P.: Compositional models in valuation-based systems. Int. J. Approx. Reasoning 53(8), 1155–1167 (2012)
Kolmogorov, A.N.: Tri podchoda k opredeleniju ponjatija ’kolichestvo informacii’. Problemy Peredachi Informacii 1, 3–11 (1965)
Kullback, S., Leibler, R.A.: On information and sufficiency. Annals of Mathematical Statistics 22, 76–86 (1951)
Lam, W., Bacchus, F.: Learning Bayesian Belief Networks: An approach based on the MDL Principle. Computational Intelligence 10, 269–293 (1994). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.127.5504&rep=rep1&type=pdf
Lauritzen, S.L.: Graphical models. Oxford University Press (1996)
Mahdi, O.A., Mohammed, M.A., Mohamed, A.J.: Implementing a Novel Approach an Convert Audio Compression to Text Coding via Hybrid Technique. Int. J. Computer Science 3(6), 53–9 (2012)
Von Mises, R.: Probability, statistics, and truth. Courier Corporation, Mineola (1957). [Originaly published in German by Springer, 1928]
Witten, I.H., Neal, R.M., Cleary, J.G.: Arithmetic Coding for Data Compression. Communications of the ACM 30(6), 520–540 (1987)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Jiroušek, R., Krejčová, I. (2015). Minimum Description Length Principle for Compositional Model Learning. In: Huynh, VN., Inuiguchi, M., Demoeux, T. (eds) Integrated Uncertainty in Knowledge Modelling and Decision Making. IUKM 2015. Lecture Notes in Computer Science(), vol 9376. Springer, Cham. https://doi.org/10.1007/978-3-319-25135-6_25
Download citation
DOI: https://doi.org/10.1007/978-3-319-25135-6_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-25134-9
Online ISBN: 978-3-319-25135-6
eBook Packages: Computer ScienceComputer Science (R0)