Abstract
We transform probabilistic programs to run more efficiently and read more easily, by composing three semantics-preserving transformations: (1) apply the denotational semantics; (2) improve the resulting integral; then (3) invert the denotational semantics. Whereas step 1 is a straightforward transformation from monadic to continuation-passing style, the rest builds on computer algebra: step 2 reorders and performs integrals, and step 3 represents density functions as differential operators.
Thanks to Mike Kucera, Praveen Narayanan, Natalie Perna, and Robert Zinkov for developing the Hakaru probabilistic programming system, the home of our research. Thanks also to the anonymous referees for many comments, which improved this paper. This research was supported by DARPA grant FA8750-14-2-0007, NSF grant CNS-0723054, Lilly Endowment, Inc. (through its support for the Indiana University Pervasive Technology Institute), and the Indiana METACyt Initiative. The Indiana METACyt Initiative at IU is also supported in part by Lilly Endowment, Inc.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
More precisely, a D-finite expression. We speak of holonomicity, which encompasses D-finite functions and P-recursive sequences, because we foresee recognizing discrete measures using P-recursive equations. Those are almost as effective as D-finite equations, modulo the issue of accurate summation (Abramov and Petkovšek 2005).
- 2.
The version of gfun shipped with Maple has several important bugs, fixed in the version at http://perso.ens-lyon.fr/bruno.salvy/software/the-gfun-package/.
- 3.
We only consider the difference between the two degrees, because the numerator and denominator may not be relatively prime, like for \(\mathsf{Beta}(1,\beta )\) and \(\mathsf{Beta}(\alpha ,1)\).
- 4.
- 5.
References
Abramov, S.A., Petkovšek, M.: Gosper’s algorithm, accurate summation, and the discrete Newton-Leibniz formula. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation (ISSAC), pp. 5–12. ACM Press (2005)
Andrews, D.F., Stafford, J.E.H.: Symbolic Computation for Statistical Inference. Oxford University Press, Oxford (2000)
Carette, J.: Understanding expression simplification. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation (ISSAC), pp. 72–79. ACM Press (2004)
Carette, J.: A canonical form for piecewise defined functions. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation (ISSAC), pp. 77–84. ACM Press (2007)
Carette, J., Shan, C.: Simplifying probabilistic programs using computer algebra. Technical report 719, Indiana University (2015). http://www.cs.indiana.edu/cgi-bin/techreports/TRNNN.cgi?trnum=TR719
Chyzak, F., Salvy, B.: Non-commutative elimination in ore algebras proves multivariate holonomic identities. J. Symb. Comput. 26(2), 187–227 (1998)
Danvy, O., Filinski, A.: Abstracting control. In: Proceedings of the Conference on LISP and Functional Programming, pp. 151–160. ACM Press (1990)
de Salvo Braz, R., Amir, E., Roth, D.: Lifted first-order probabilistic inference. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, pp. 1319–1325 (2005)
Dechter, R.: Bucket elimination: a unifying framework for probabilistic inference. In: Jordan, M.I., (ed.) Learning and Inference in Graphical Models. Kluwer, Dordrecht, Paperback: Learning in Graphical Models. MIT Press (1998)
Giry, M.: A categorical approach to probability theory. In: Banaschewski, B. (ed.) Categorical Aspects of Topology and Analysis. Lecture Notes in Mathematics, vol. 915, pp. 68–85. Springer, Heidelberg (1981)
Hatcliff, J., Danvy, O.: A generic account of continuation-passing styles. In: Proceedings of the 21st Symposium on Principles of Programming Languages (POPL), pp. 458–471. ACM Press (1994)
Kauers, M.: The holonomic toolkit. In: Schneider, C., Blümlein, J. (eds.) Computer Algebra in Quantum Field Theory. Texts and Monographs in Symbolic Computation, pp. 119–144. Springer, Vienna (2013)
Maybeck, P.S.: Stochastic Models, Estimation, and Control. Academic Press, New York (1979)
Moses, J.: Algebraic simplification: a guide for the perplexed. Commun. ACM 14(8), 548–560 (1971)
Pollard, D.: A User’s Guide to Measure Theoretic Probability. Cambridge University Press, Cambridge (2001)
Ramsey, N., Pfeffer, A.: Stochastic lambda calculus and monads of probability distributions. In: Proceedings of the 29th Symposium on Principles of Programming Languages (POPL), pp. 154–165. ACM Press (2002)
Salvy, B.: D-finiteness: Algorithms and applications. In: Proceedings of the International Symposium on Symbolic and Algebraic Computation (ISSAC), pp. 2–3 (2005)
Salvy, B., Zimmermann, P.: Gfun: a Maple package for the manipulation of generating and holonomic functions in one variable. ACM Trans. Math. Softw. 20(2), 163–177 (1994)
Tristan, J.B., Huang, D., Tassarotti, J., Pocock, A.C., Green, S.J., Steele, G.L., Jr.: Augur: Data-parallel probabilistic modeling. In: Advances in Neural Information Processing Systems, pp. 2600–2608 (2014)
Wilf, H.S., Zeilberger, D.: An algorithmic proof theory for hypergeometric (ordinary and “q”) multisum/integral identities. Inventiones mathematicae 108, 557–633 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Carette, J., Shan, CC. (2016). Simplifying Probabilistic Programs Using Computer Algebra. In: Gavanelli, M., Reppy, J. (eds) Practical Aspects of Declarative Languages. PADL 2016. Lecture Notes in Computer Science(), vol 9585. Springer, Cham. https://doi.org/10.1007/978-3-319-28228-2_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-28228-2_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28227-5
Online ISBN: 978-3-319-28228-2
eBook Packages: Computer ScienceComputer Science (R0)