Skip to main content
SpringerLink
Log in
Book cover

JSAI International Symposium on Artificial Intelligence

JSAI-isAI 2010: New Frontiers in Artificial Intelligence pp 167–180Cite as

Relax, Compensate and Then Recover

  • Conference paper

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 6797))

Abstract

We present in this paper a framework of approximate probabilistic inference which is based on three simple concepts. First, our notion of an approximation is based on “relaxing” equality constraints, for the purposes of simplifying a problem so that it can be solved more readily. Second, is the concept of “compensation,” which calls for imposing weaker notions of equality to compensate for the relaxed equality constraints. Third, is the notion of “recovery,” where some of the relaxed equality constraints are incrementally recovered, based on an assessment of their impact on improving the quality of an approximation. We discuss how this framework subsumes one of the most influential algorithms in probabilistic inference: loopy belief propagation and some of its generalizations. We also introduce a new heuristic recovery method that was key to a system that successfully participated in a recent evaluation of approximate inference systems, held in UAI’10. We further discuss the relationship between this framework for approximate inference and an approach to exact inference based on symbolic reasoning.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Aji, S.M., McEliece, R.J.: The generalized distributive law and free energy minimization. In: Proceedings of the 39th Allerton Conference on Communication, Control and Computing, pp. 672–681 (2001)

    Google Scholar 

  2. Bilmes, J.: Results from the evaluation of probabilistic inference systems at UAI 2006 (2006), http://ssli.ee.washington.edu/~bilmes/uai06InferenceEvaluation/results

  3. Chavira, M., Darwiche, A.: On probabilistic inference by weighted model counting. Artificial Intelligence 172(6-7), 772–799 (2008)

    Article  MathSciNet  Google Scholar 

  4. Chavira, M., Darwiche, A., Jaeger, M.: Compiling relational Bayesian networks for exact inference. International Journal of Approximate Reasoning 42(1-2), 4–20 (2006)

    Article  MathSciNet  Google Scholar 

  5. Choi, A., Chan, H., Darwiche, A.: On Bayesian network approximation by edge deletion. In: Proceedings of the 21st Conference on Uncertainty in Artificial Intelligence (UAI), pp. 128–135. Arlington, Virginia (2005)

    Google Scholar 

  6. Choi, A., Chavira, M., Darwiche, A.: Node splitting: A scheme for generating upper bounds in Bayesian networks. In: Proceedings of the 23rd Conference on Uncertainty in Artificial Intelligence (UAI), pp. 57–66 (2007)

    Google Scholar 

  7. Choi, A., Darwiche, A.: An edge deletion semantics for belief propagation and its practical impact on approximation quality. In: Proceedings of the 21st National Conference on Artificial Intelligence (AAAI), pp. 1107–1114 (2006)

    Google Scholar 

  8. Choi, A., Darwiche, A.: A variational approach for approximating Bayesian networks by edge deletion. In: Proceedings of the 22nd Conference on Uncertainty in Artificial Intelligence (UAI), pp. 80–89 (2006)

    Google Scholar 

  9. Choi, A., Darwiche, A.: Approximating the partition function by deleting and then correcting for model edges. In: Proceedings of the 24th Conference on Uncertainty in Artificial Intelligence (UAI), pp. 79–87 (2008)

    Google Scholar 

  10. Choi, A., Darwiche, A.: Focusing generalizations of belief propagation on targeted queries. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), pp. 1024–1030 (2008)

    Google Scholar 

  11. Choi, A., Darwiche, A.: Many-pairs mutual information for adding structure to belief propagation approximations. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence (AAAI), pp. 1031–1036 (2008)

    Google Scholar 

  12. Choi, A., Darwiche, A.: Relax then compensate: On max-product belief propagation and more. In: Proceedings of the Twenty-Third Annual Conference on Neural Information Processing Systems (NIPS), pp. 351–359 (2009)

    Google Scholar 

  13. Choi, A., Standley, T., Darwiche, A.: Approximating weighted Max-SAT problems by compensating for relaxations. In: Gent, I.P. (ed.) CP 2009. LNCS, vol. 5732, pp. 211–225. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  14. Darwiche, A.: Decomposable negation normal form. Journal of the ACM 48(4), 608–647 (2001)

    Article  MathSciNet  Google Scholar 

  15. Darwiche, A.: Recursive conditioning. Artificial Intelligence 126(1-2), 5–41 (2001)

    Article  MathSciNet  Google Scholar 

  16. Darwiche, A.: A compiler for deterministic, decomposable negation normal form. In: Proceedings of the Eighteenth National Conference on Artificial Intelligence (AAAI), pp. 627–634. AAAI Press, Menlo Park (2002)

    Google Scholar 

  17. Darwiche, A.: A logical approach to factoring belief networks. In: Proceedings of KR, pp. 409–420 (2002)

    Google Scholar 

  18. Darwiche, A.: A differential approach to inference in Bayesian networks. Journal of the ACM 50(3), 280–305 (2003)

    Article  MathSciNet  Google Scholar 

  19. Darwiche, A., Dechter, R., Choi, A., Gogate, V., Otten, L.: Results from the probablistic inference evaluation of UAI 2008 (2008), http://graphmod.ics.uci.edu/uai08/Evaluation/Report

  20. Darwiche, A., Marquis, P.: A knowledge compilation map. Journal of Artificial Intelligence Research 17, 229–264 (2002)

    Article  MathSciNet  Google Scholar 

  21. Dechter, R.: Bucket elimination: A unifying framework for probabilistic inference. In: Proceedings of the 12th Conference on Uncertainty in Artificial Intelligence (UAI), pp. 211–219 (1996)

    Google Scholar 

  22. Dechter, R., Rish, I.: Mini-buckets: A general scheme for bounded inference. J. ACM 50(2), 107–153 (2003)

    Article  MathSciNet  Google Scholar 

  23. Elidan, G., Globerson, A.: Summary of the 2010 UAI approximate inference challenge (2010), http://www.cs.huji.ac.il/project/UAI10/summary.php

  24. Elidan, G., McGraw, I., Koller, D.: Residual belief propagation: Informed scheduling for asynchronous message passing. In: Proceedings of the 22nd Conference in Uncertainty in Artificial Intelligence (2006)

    Google Scholar 

  25. Geiger, D., Meek, C., Wexler, Y.: A variational inference procedure allowing internal structure for overlapping clusters and deterministic constraints. J. Artif. Intell. Res. (JAIR) 27, 1–23 (2006)

    Article  MathSciNet  Google Scholar 

  26. Jaakkola, T.: Tutorial on variational approximation methods. In: Saad, D., Opper, M. (eds.) Advanced Mean Field Methods, ch. 10, pp. 129–160. MIT Press, Cambridge (2001)

    Google Scholar 

  27. Jensen, F.V., Lauritzen, S., Olesen, K.: Bayesian updating in recursive graphical models by local computation. Computational Statistics Quarterly 4, 269–282 (1990)

    MATH  Google Scholar 

  28. Jha, A., Suciu, D.: Knowledge compilation meets database theory: compiling queries to decision diagrams. In: Proceedings of the 14th International Conference on Database Theory, pp. 162–173 (2011)

    Google Scholar 

  29. Jordan, M.I., Ghahramani, Z., Jaakkola, T., Saul, L.K.: An introduction to variational methods for graphical models. Machine Learning 37(2), 183–233 (1999)

    Article  Google Scholar 

  30. Lauritzen, S.L., Spiegelhalter, D.J.: Local computations with probabilities on graphical structures and their application to expert systems. Journal of Royal Statistics Society, Series B 50(2), 157–224 (1988)

    MathSciNet  MATH  Google Scholar 

  31. Mateescu, R., Kask, K., Gogate, V., Dechter, R.: Join-graph propagation algorithms. J. Artif. Intell. Res. (JAIR) 37, 279–328 (2010)

    Article  MathSciNet  Google Scholar 

  32. Mooij, J.M., Kappen, H.J.: Sufficient conditions for convergence of the sum-product algorithm. IEEE Transactions on Information Theory 53(12), 4422–4437 (2007)

    Article  MathSciNet  Google Scholar 

  33. Pearl, J.: Probabilistic Reasoning in Intelligent Systems: Networks of Plausible Inference. Morgan Kaufmann, San Francisco (1988)

    MATH  Google Scholar 

  34. Pipatsrisawat, K., Darwiche, A.: New compilation languages based on structured decomposability. In: Proceedings of the Twenty-Third AAAI Conference on Artificial Intelligence (AAAI), pp. 517–522 (2008)

    Google Scholar 

  35. Pipatsrisawat, K., Darwiche, A.: Top-down algorithms for constructing structured DNNF: Theoretical and practical implications. In: Proceedings of the 19th European Conference on Artificial Intelligence, pp. 3–8 (2010)

    Google Scholar 

  36. Yedidia, J.S., Freeman, W.T., Weiss, Y.: Understanding belief propagation and its generalizations. In: Lakemeyer, G., Nebel, B. (eds.) Exploring Artificial Intelligence in the New Millennium, ch. 8, pp. 239–269. Morgan Kaufmann, San Francisco (2003)

    Google Scholar 

  37. Yuille, A.L.: CCCP algorithms to minimize the Bethe and Kikuchi free energies: Convergent alternatives to belief propagation. Neural Computation 14(7), 1691–1722 (2002)

    Article  Google Scholar 

  38. Zhang, N.L., Poole, D.: Exploiting causal independence in Bayesian network inference. Journal of Artificial Intelligence Research 5, 301–328 (1996)

    Article  MathSciNet  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Computer Science Department, University of California, Los Angeles, USA

    Arthur Choi & Adnan Darwiche

Authors
  1. Arthur Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Adnan Darwiche
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. System Engineering Research Lab., CRIEPI, 2-11-1, Iwado Kita, 201-8511, Komae-shi, Tokyo, Japan

    Takashi Onada

  2. Graduate School of Humanities and Sciences, Faculty of Science, Department of Information Science, Ochanomizu University, Otsuka 2-1-1, 112-8610, Bunkyo-ku, Tokyo, Japan

    Daisuke Bekki

  3. Department of English, 4-4-25 Shibuya, Aoyama Gakuin University, 150-8366, Shibuya-ku, Tokyo, Japan

    Elin McCready

Rights and permissions

Reprints and permissions

Copyright information

© 2011 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Choi, A., Darwiche, A. (2011). Relax, Compensate and Then Recover. In: Onada, T., Bekki, D., McCready, E. (eds) New Frontiers in Artificial Intelligence. JSAI-isAI 2010. Lecture Notes in Computer Science(), vol 6797. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25655-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-25655-4_16

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-25654-7

  • Online ISBN: 978-3-642-25655-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation