Skip to main content
SpringerLink
Log in

Incremental Analysis for Probabilistic Programs

  • Conference paper
  • First Online:
Static Analysis (SAS 2017)

Part of the book series: Lecture Notes in Computer Science ((LNPSE,volume 10422))

Included in the following conference series:

Abstract

This paper presents Icpp, a new data-flow-based InCremental analysis for Probabilistic Programs, to infer their posterior probability distributions in response to small yet frequent changes to probabilistic knowledge, i.e., prior probability distributions and observations. Unlike incremental analyses for usual programs, which emphasize code changes, such as statement additions and deletions, Icpp focuses on changes made to probabilistic knowledge, the key feature in probabilistic programming. The novelty of Icpp lies in capturing the correlation between prior and posterior probability distributions by reasoning about the probabilistic dependence of each data-flow fact, so that any posterior probability affected by newly changed probabilistic knowledge can be incrementally updated in a sparse manner without recomputing it from scratch, thereby allowing the previously computed results to be reused. We have evaluated Icpp with a set of probabilistic programs. Our results show that Icpp is an order of magnitude faster than the state-of-the-art data-flow-based inference in analyzing probabilistic programs under small yet frequent changes to probabilistic knowledge, with an average analysis overhead of around 0.1 s in response to a single change.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Arzt, S., Bodden, E.: Reviser: efficiently updating IDE-/IFDS-based data-flow analyses in response to incremental program changes. In: Proceedings of the 36th International Conference on Software Engineering, pp. 288–298 (2014)

    Google Scholar 

  2. Borgström, J., Gordon, A.D., Greenberg, M., Margetson, J., Gael, J.: Measure transformer semantics for Bayesian machine learning. In: Barthe, G. (ed.) ESOP 2011. LNCS, vol. 6602, pp. 77–96. Springer, Heidelberg (2011). doi:10.1007/978-3-642-19718-5_5

    Chapter  Google Scholar 

  3. Caticha, A., Giffin, A., Mohammad-Djafari, A.: Updating probabilities. In: 18th International Conference on Artificial Intelligence and Pattern Recognition, pp. 31–42 (2006)

    Google Scholar 

  4. Chaganty, A.T., Nori, A.V., Rajamani, S.K.: Efficiently sampling probabilistic programs via program analysis. In: 16th Artificial Intelligence and Statistics, pp. 153–160 (2013)

    Google Scholar 

  5. Chakarov, A., Sankaranarayanan, S.: Expectation invariants for probabilistic program loops as fixed points. In: Müller-Olm, M., Seidl, H. (eds.) SAS 2014. LNCS, vol. 8723, pp. 85–100. Springer, Cham (2014). doi:10.1007/978-3-319-10936-7_6

    Google Scholar 

  6. Chan, H., Darwiche, A.: On the revision of probabilistic beliefs using uncertain evidence. Artif. Intell. 163(1), 67–90 (2005)

    Google Scholar 

  7. Chib, S., Greenberg, E.: Understanding the metropolis-hastings algorithm. Am. Stat. 49(4), 327–335 (1995)

    Google Scholar 

  8. Claret, G., Rajamani, S.K., Nori, A.V., Gordon, A.D., Borgström, J.: Bayesian inference using data flow analysis. In: Proceedings of the 9th Joint Meeting on Foundations of Software Engineering, pp. 92–102 (2013)

    Google Scholar 

  9. Fan, X., Sui, Y., Liao, X., Xue, J.: Boosting the precision of virtual call integrity protection with partial pointer analysis for C++. In: 26th ACM SIGSOFT International Symposium on Software Testing and Analysis (2017)

    Google Scholar 

  10. Gehr, T., Misailovic, S., Vechev, M.: PSI: exact symbolic inference for probabilistic programs. In: 26th International Conference on Computer Aided Verification, pp. 62–83 (2016)

    Google Scholar 

  11. Glynn, P.W., Iglehart, D.L.: Importance sampling for stochastic simulations. Manag. Sci. 35(11), 1367–1392 (1989)

    Google Scholar 

  12. Gordon, A.D., Henzinger, T.A., Nori, A.V., Rajamani, S.K.: Probabilistic programming. In: Proceedings of the on Future of Software Engineering, pp. 167–181 (2014)

    Google Scholar 

  13. Hastings, W.K.: Monte Carlo sampling methods using Markov chains and their applications. Biometrika 57(1), 97–109 (1970)

    Google Scholar 

  14. Hermenegildo, M., Puebla, G., Marriott, K., Stuckey, P.J.: Incremental analysis of constraint logic programs. ACM Trans. Program. Lang. Syst. 22(2), 187–223 (2000)

    Google Scholar 

  15. Hur, C.-K., Nori, A.V., Rajamani, S.K., Samuel, S.: Slicing probabilistic programs. In: Proceedings of the 35th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 133–144 (2014)

    Google Scholar 

  16. Koller, D., Friedman, N.: Probabilistic Graphical Models: Principles and Techniques. The MIT Press, Cambridge (2009)

    Google Scholar 

  17. Kullback, S.: Information Theory and Statistics. Dover Publications, New York (1997)

    Google Scholar 

  18. Lu, Y., Shang, L., Xie, X., Xue, J.: An incremental points-to analysis with CFL-reachability. In: Jhala, R., Bosschere, K. (eds.) CC 2013. LNCS, vol. 7791, pp. 61–81. Springer, Heidelberg (2013). doi:10.1007/978-3-642-37051-9_4

    Chapter  Google Scholar 

  19. MacKay, D.J.: Introduction to Monte Carlo methods. Learning in Graphical Models, vol. 89, pp. 175–204. Springer, Berlin (1998)

    Google Scholar 

  20. Mardziel, P., Magill, S., Hicks, M., Srivatsa, M.: Dynamic enforcement of knowledge-based security policies using probabilistic abstract interpretation. J. Comput. Secur. 21(4), 463–532 (2013)

    Google Scholar 

  21. Miller, A.C., Rice, T.R.: Discrete approximations of probability distributions. Manag. Sci. 29(3), 352–362 (1983)

    Google Scholar 

  22. Minka, T.P.: Expectation propagation for approximate Bayesian inference. In: Proceedings of the 17th Conference on Uncertainty in Artificial Intelligence, pp. 362–369 (2001)

    Google Scholar 

  23. Monniaux, D.: Abstract interpretation of probabilistic semantics. In: Palsberg, J. (ed.) SAS 2000. LNCS, vol. 1824, pp. 322–339. Springer, Heidelberg (2000). doi:10.1007/978-3-540-45099-3_17

    Chapter  Google Scholar 

  24. Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy belief propagation for approximate inference: an empirical study. In: Proceedings of the 15th Conference on Uncertainty in Artificial Intelligence, pp. 467–475 (1999)

    Google Scholar 

  25. Nori, A.V., Hur, C.-K., Rajamani, S.K., Samuel, S.: R2: an efficient MCMC sampler for probabilistic programs. In: Proceedings of the 29th National Conference on Artificial Intelligence, pp. 2476–2482 (2014)

    Google Scholar 

  26. Person, S., Yang, G., Rungta, N., Khurshid, S.: Directed incremental symbolic execution. In: Proceedings of the 32nd ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 504–515 (2011)

    Google Scholar 

  27. Pfeffer, A.: Figaro: an object-oriented probabilistic programming language. Charles River Analytics Technical report 137:96 (2009)

    Google Scholar 

  28. Plummer, M., et al.: Jags: a program for analysis of Bayesian graphical models using Gibbs sampling. In: Proceedings of the 3rd International Workshop on Distributed Statistical Computing, p. 125 (2003)

    Google Scholar 

  29. Sankaranarayanan, S., Chakarov, A., Gulwani, S.: Static analysis for probabilistic programs: inferring whole program properties from finitely many paths. In: Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation, pp. 447–458 (2013)

    Google Scholar 

  30. Sanner, S., McAllester, D.: Affine algebraic decision diagrams (AADDs) and their application to structured probabilistic inference. In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, pp. 1384–1390 (2005)

    Google Scholar 

  31. Shang, L., Lu, Y., Xue, J.: Fast and precise points-to analysis with incremental CFL-reachability summarisation: preliminary experience. In: Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering, pp. 270–273 (2012)

    Google Scholar 

  32. Sui, Y., Xue, J.: On-demand strong update analysis via value-flow refinement. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 460–473 (2016)

    Google Scholar 

  33. Sui, Y., Xue, J.: SVF: interprocedural static value-flow analysis in LLVM. In: Proceedings of the 25th International Conference on Compiler Construction, pp. 265–266 (2016)

    Google Scholar 

  34. Vallée-Rai, R., Co, P., Gagnon, E., Hendren, L., Lam, P., Sundaresan, V.: Soot - a Java bytecode optimization framework. In: Proceedings of the Conference of the Centre for Advanced Studies on Collaborative Research, p. 13 (1999)

    Google Scholar 

  35. Wainwright, M.J., Jordan, M.I., et al.: Graphical models, exponential families, and variational inference. Found. Trends® Mach. Learn. 1(1–2), 1–305 (2008)

    Google Scholar 

  36. Wanke, C., Greenbaum, D.: Incremental, probabilistic decision making for en route traffic management. Air Traffic Cont. Q. 15(4), 299–319 (2007)

    Google Scholar 

  37. Yue, A., Liu, W.: Revising imprecise probabilistic beliefs in the framework of probabilistic logic programming. In: Proceedings of the 23rd National Conference on Artificial Intelligence, pp. 590–596 (2008)

    Google Scholar 

  38. Zhan, S., Huang, J.: ECHO: instantaneous in situ race detection in the IDE. In: Proceedings of the 24th ACM SIGSOFT International Symposium on Foundations of Software Engineering, pp. 775–786 (2016)

    Google Scholar 

Download references

Acknowledgments

The authors wish to thank the anonymous reviewers for their valuable comments. The first author would like to thank Aleksandar Chakarov for some helpful email discussions. This work is supported by Australian Research Grants, DP150102109, DP170103956 and DE170101081.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, UNSW, Sydney, Australia

    Jieyuan Zhang, Yulei Sui & Jingling Xue

  2. Advanced Innovation Center for Imaging Technology, CNU, Beijing, China

    Jingling Xue

Authors
  1. Jieyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yulei Sui
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jingling Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jieyuan Zhang .

Editor information

Editors and Affiliations

  1. Dipartimento di Matematica, University of Padova, Padua, Italy

    Francesco Ranzato

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Zhang, J., Sui, Y., Xue, J. (2017). Incremental Analysis for Probabilistic Programs. In: Ranzato, F. (eds) Static Analysis. SAS 2017. Lecture Notes in Computer Science(), vol 10422. Springer, Cham. https://doi.org/10.1007/978-3-319-66706-5_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-66706-5_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-66705-8

  • Online ISBN: 978-3-319-66706-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation