Abstract
We present AQUA, a new probabilistic inference algorithm that operates on probabilistic programs with continuous posterior distributions. AQUA approximates programs via an efficient quantization of the continuous distributions. It represents the distributions of random variables using quantized value intervals (Interval Cube) and corresponding probability densities (Density Cube). AQUA’s analysis transforms Interval and Density Cubes to compute the posterior distribution with bounded error. We also present an adaptive algorithm for selecting the size and the granularity of the Interval and Density Cubes.
We evaluate AQUA on 24 programs from the literature. AQUA solved all of 24 benchmarks in less than 43 s (median 1.35 s) with a high-level of accuracy. We show that AQUA is more accurate than state-of-the-art approximate algorithms (Stan’s NUTS and ADVI) and supports programs that are out of reach of exact inference tools, such as PSI and SPPL.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We support common continuous distributions including Normal, Uniform, Exponential, Beta, Gamma, Student-T, Laplace, Triangular, and any mixture of the above distributions.
References
Aguirre, A., Barthe, G., Hsu, J., Kaminski, B.L., Katoen, J.P., Matheja, C.: A pre-expectation calculus for probabilistic sensitivity. POPL (2021)
Albarghouthi, A., D’Antoni, L., Drews, S., Nori, A.: Fairsquare: probabilistic verification of program fairness. OOPSLA (2017)
Bissiri, P., Holmes, C., Walker, S.: A general framework for updating belief distributions. J. R. Stat. Soc. Ser. B Stat. Methodol. 78(5), 1103 (2016)
Borges, M., Filieri, A., d’Amorim, M., Păsăreanu, C.S., Visser, W.: Compositional solution space quantification for probabilistic software analysis. PLDI (2014)
Carpenter, B., Gelman, A., Hoffman, M., Lee, D., et al.: Stan: a probabilistic programming language. JSTATSOFT 20(2) (2016)
Dutta, S., Legunsen, O., Huang, Z., Misailovic, S.: Testing probabilistic programming systems. In: FSE (2018)
Dutta, S., Zhang, W., Huang, Z., Misailovic, S.: Storm: program reduction for testing and debugging probabilistic programming systems. In: FSE (2019)
Gehr, T., Misailovic, S., Vechev, M.: PSI: exact symbolic inference for probabilistic programs. In: Chaudhuri, S., Farzan, A. (eds.) CAV 2016. LNCS, vol. 9779, pp. 62–83. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-41528-4_4
Gelman, A., Stern, H.S., Carlin, J.B., Dunson, D.B., Vehtari, A., Rubin, D.B.: Bayesian Data Analysis. Chapman and Hall/CRC (2013)
Goodman, N., Tenenbaum, J.: Probabilistic Models of Cognition. http://probmods.org/
Gorinova, M.I., Gordon, A.D., Sutton, C.: Probabilistic programming with densities in SlicStan: efficient, flexible, and deterministic. POPL (2019)
Holtzen, S., Van den Broeck, G., Millstein, T.: Scaling exact inference for discrete probabilistic programs. OOPSLA (2020)
Huang, Z., Wang, Z., Misailovic, S.: PSense: automatic sensitivity analysis for probabilistic programs. In: Lahiri, S.K., Wang, C. (eds.) ATVA 2018. LNCS, vol. 11138, pp. 387–403. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01090-4_23
Laurel, J., Misailovic, S.: Continualization of probabilistic programs with correction. ESOP (2020)
Luo, Y., Filieri, A., Zhou, Y.: SYMPAIS: symbolic parallel adaptive importance sampling for probabilistic program analysis. arXiv preprint arXiv:2010.05050 (2020)
Narayanan, P., Carette, J., Romano, W., Shan, C., Zinkov, R.: Probabilistic inference by program transformation in Hakaru (system description). In: Kiselyov, O., King, A. (eds.) FLOPS 2016. LNCS, vol. 9613, pp. 62–79. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-29604-3_5
Neal, R.M.: Bayesian Learning for Neural Networks. Springer, Heidelberg (2012)
Nishihara, R., Minka, T., Tarlow, D.: Detecting parameter symmetries in probabilistic models. arXiv preprint arXiv:1312.5386 (2013)
Saad, F.A., Rinard, M.C., Mansinghka, V.K.: SPPL: a probabilistic programming system with exact and scalable symbolic inference. PLDI (2021)
Sankaranarayanan, S., Chakarov, A., Gulwani, S.: Static analysis for probabilistic programs: Inferring whole program properties from finitely many paths. PLDI (2013)
Sweet, I., Trilla, J.M.C., Scherrer, C., Hicks, M., Magill, S.: What’s the over/under? probabilistic bounds on information leakage. POST (2018)
Wang, C., Blei, D.M.: A general method for robust Bayesian modeling. Bayesian Anal. 13(4), 1159–1187 (2018)
Wang, Y., Kucukelbir, A., Blei, D.M.: Robust probabilistic modeling with Bayesian data reweighting. ICML (2017)
Acknowledgements
This research was supported in part by NSF Grants No. CCF-1846354, CCF-1956374, CCF-2008883, and Facebook PhD Fellowship.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Huang, Z., Dutta, S., Misailovic, S. (2021). AQUA: Automated Quantized Inference for Probabilistic Programs. In: Hou, Z., Ganesh, V. (eds) Automated Technology for Verification and Analysis. ATVA 2021. Lecture Notes in Computer Science(), vol 12971. Springer, Cham. https://doi.org/10.1007/978-3-030-88885-5_16
Download citation
DOI: https://doi.org/10.1007/978-3-030-88885-5_16
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-88884-8
Online ISBN: 978-3-030-88885-5
eBook Packages: Computer ScienceComputer Science (R0)