Abstract
Application of the Minimum Description Length principle to optimization queries in probabilistic programming was investigated on the example of the C++ probabilistic programming library under development. It was shown that incorporation of this criterion is essential for optimization queries to behave similarly to more common queries performing sampling in accordance with posterior distributions and automatically implementing the Bayesian Occam’s razor. Experimental validation was conducted on the task of blood cell detection on microscopic images. Detection appeared to be possible using genetic programming query, and automatic penalization of candidate solution complexity allowed to choose the number of cells correctly avoiding overfitting.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Hutter, M.: Universal Artificial Intelligence: Sequential Decisions Based on Algorithmic Probability. Springer (2005)
Wallace, C.S., Boulton, D.M.: An Information Measure for Classification. Computer Journal 11, 185–195 (1968)
Rissanen, J.J.: Modeling by the Shortest Data Description. Automatica-J.IFAC 14, 465–471 (1978)
Vitanyi, P.M.B., Li, M.: Minimum Description Length Induction, Bayesianism, and Kolmogorov complexity. IEEE Trans. on Information Theory 46(2), 446–464 (2000)
Potapov, A.S.: Principle of Representational Minimum Description Length in Image Analysis and Pattern Recognition. Pattern Recognition and Image Analysis 22(1), 82–91 (2012)
Solomonoff, R.: Does Algorithmic Probability Solve the Problem of Induction?. Oxbridge Research, Cambridge (1997)
MacKay, D.J.C.: Bayesian Methods for Adaptive Models. PhD thesis, California Institute of Technology (1991)
Goodman, N.D., Tenenbaum, J.B.: Probabilistic Models of Cognition. https://probmods.org/
Stuhlmüller, A., Goodman, N. D.: A dynamic programming algorithm for inference in recursive probabilistic programs. In: Second Statistical Relational AI Workshop at UAI 2012 (StaRAI-12), arXiv:1206.3555 [cs.AI] (2012)
Chaganty, A., Nori A.V., Rajamani, S.K.: Efficiently sampling probabilistic programs via program analysis. In: Proc. Artificial Intelligence and Statistics, pp. 153–160 (2013)
Potapov, A., Batishcheva, V.: Genetic Programming on Program Traces as an Inference Engine for Probabilistic Languages. In: LNAI (2015)
Mansinghka, V., Kulkarni, T., Perov, Y., Tenenbaum, J.: Approximate Bayesian Image Interpretation using Generative Probabilistic Graphics Programs. Advances in Neural Information Processing Systems, arXiv:1307.0060 [cs.AI] (2013)
Zhdanov, I.N., Potapov, A.S., Shcherbakov, O.V.: Erythrometry method based on a modified Hough transform. Journal of Optical Technology 80(3), 201–203 (2013)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Potapov, A., Batishcheva, V., Rodionov, S. (2015). Optimization Framework with Minimum Description Length Principle for Probabilistic Programming. In: Bieger, J., Goertzel, B., Potapov, A. (eds) Artificial General Intelligence. AGI 2015. Lecture Notes in Computer Science(), vol 9205. Springer, Cham. https://doi.org/10.1007/978-3-319-21365-1_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-21365-1_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21364-4
Online ISBN: 978-3-319-21365-1
eBook Packages: Computer ScienceComputer Science (R0)