Skip to main content
SpringerLink
Log in

On the Minima of Bethe Free Energy in Gaussian Distributions

  • Conference paper
Artificial Intelligence and Soft Computing – ICAISC 2008 (ICAISC 2008)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 5097))

Included in the following conference series:

  • 1560 Accesses

Abstract

Belief propagation (BP) is effective for computing marginal probabilities of a high dimensional probability distribution. Loopy belief propagation (LBP) is known not to compute precise marginal probabilities and not to guarantee its convergence. The fixed points of LBP are known to accord with the extrema of Bethe free energy. Hence, the fixed points are analyzed by minimizing the Bethe free energy.

In this paper, we consider the Bethe free energy in Gaussian distributions and analytically clarify the extrema, equivalently, the fixed points of LBP for some particular cases. The analytical results tell us a necessary condition for LBP convergence and the quantities which determine the accuracy of LBP in Gaussian distributions. Based on the analytical results, we perform numerical experiments of LBP and compare the results with analytical solutions.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Heskes, T.: On the uniqueness of loopy belief propagation fixed points. Neural Computation 16(11), 2379–2414 (2004)

    Article  MATH  Google Scholar 

  2. Weiss, Y.: Belief propagation and revision in networks with loops. Technical report 1616, MIT AI lab (1997)

    Google Scholar 

  3. Weiss, Y.: Correctness of local probability propagation in graphical models with loops. Neural Computation 12(1), 1–41 (2000)

    Article  MATH  Google Scholar 

  4. Weiss, Y., Freeman, W.: Correctness of belief propagation in gaussian graphical models of arbitrary topology. Neural Computation 13(10), 2173–2200 (2001)

    Article  MATH  Google Scholar 

  5. Ikeda, S., Tanaka, T., Amari, S.: Stochastic reasoning, free energy, and information geometry. Neural Computation 16(9), 1779–1810 (2004)

    Article  MATH  Google Scholar 

  6. Ikeda, S., Tanaka, T., Amari, S.: Information geometry of turbo and low-density parity-check codes. IEEE Trans. Inf. Theory 50(6), 1097–1114 (2004)

    Article  MathSciNet  Google Scholar 

  7. Kabashima, Y., Saad, D.: The TAP approach to intensive and extensive connectivity systems. In: Opper, M., Saad, D. (eds.) Advanced Mean Field Methods -Theory and Practice, pp. 65–84. MIT Press, Cambridge (2001)

    Google Scholar 

  8. Yedidia, J., Freeman, W., Weiss, Y.: Bethe free energy, kikuchi approximations, and belief propagation algorithms. Technical Report TR2001-16, Mitsubishi Electric Research Laboratories (2001)

    Google Scholar 

  9. Tanaka, K., Shouno, H., Okada, M.: Accuracy of the bethe approximation for hyperparameter estimation in probabilistic image processing. J.Phys. A, Math. Gen. 37(36), 8675–8696 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  10. Tanaka, K., Inoue, J., Titterington, D.M.: Loopy belief propagation and probabilistic image processing. In: proceedings of 2003 IEEE International workshop on Neural Networks for Signal Processing, vol. 13, pp. 383–392 (2003)

    Google Scholar 

  11. Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Belief Propagation in Gaussian Models. IEICE Technical Report 107(50), 23–28 (2007)

    Google Scholar 

  12. Nishiyama, Y., Watanabe, S.: Theoretical Analysis of Accuracy of Gaussian Belief Propagation. In: Proceedings of International Conference on Artificial Neural Networks 2007, pp. 29–38 (2007)

    Google Scholar 

  13. Yuille, A.L.: CCCP algorithms to minimize the Bethe and Kikuchi free energies: convergent alternatives to belief propagation. Neural Computation 14(7), 1691–1722 (2002)

    Article  MATH  Google Scholar 

  14. Tanaka, K.: Generalized belief propagation formula in probabilistic information processing based on gaussian graphical model. IEICE D-II, Vol. J88-D-II, No. 12, pp. 2368–2379 (in Japanese) (2005)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computational Intelligence and Systems Science, Tokyo Institute of Technology, 4259, Nagatuta, Midori-ku, Yokohama, 226-8503, Japan

    Yu Nishiyama

  2. Precision and Intelligence Laboratory, Tokyo Institute of Technology, 4259, Nagatsuta, Midori-ku, Yokohama, 226-8503, Japan

    Sumio Watanabe

Authors
  1. Yu Nishiyama
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sumio Watanabe
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Leszek Rutkowski Ryszard Tadeusiewicz Lotfi A. Zadeh Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2008 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Nishiyama, Y., Watanabe, S. (2008). On the Minima of Bethe Free Energy in Gaussian Distributions. In: Rutkowski, L., Tadeusiewicz, R., Zadeh, L.A., Zurada, J.M. (eds) Artificial Intelligence and Soft Computing – ICAISC 2008. ICAISC 2008. Lecture Notes in Computer Science(), vol 5097. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69731-2_101

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-69731-2_101

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-69572-1

  • Online ISBN: 978-3-540-69731-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation