Abstract
Computing marginal probabilities from a high-dimensional distribution is generally an intractable task which arises in various probabilistic information processing. The minimum of Kikuchi free energy is known to yield more accurate marginals than that of Bethe free energy. Concave convex procedure (CCCP) proposed by Yuille is known to be an algorithm which guarantees to monotonically decrease both free energies.
In this paper, we generalize CCCP algorithm for Kikuchi free energy and present a new CCCP (NCCCP) algorithm which includes a high-dimensional free vector. The free vector enables NCCCP to be more stable and to reduce enormous computational cost underlying CCCP. Moreover it seems that there exists an optimal series of free vectors that can minimize the huge computational cost. This means that there exists an optimal concave and convex decomposition schedule in Kikuchi free energy.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Pearl, J.: Probabilistic Reasoning in Intelligent Systems. Morgan Kaufmann, San Mateo (1988)
Murphy, K.P., Weiss, Y., Jordan, M.I.: Loopy Belief Propagation for Approximate Inference: An Empirical Study. In: 15th conference on Uncertainty in Artificial Intelligence, pp. 467–475 (1999)
McEliece, R.J., MacKay, D.J.C., Cheng, J.: Turbo Decoding as an Instance of Pearl’s ”Belief Propagation” Algorithm. IEEE Journal on Selected Areas in Communications 16(2), 140–152 (1998)
MacKay, D.J.C.: Good Error-Correcting Codes Based on Very Sparse Matrices. IEEE Transactions on Information Theory 45(2), 399–431 (1999)
Kabashima, Y.: A CDMA multiuser detection algorithm on the basis of belief propagation. J. Phys. A, Math. Gen. 36(43), 11111–11121 (2003)
Tanaka, K.: Statistical-mechanical approach to image processing. J. Phys. A, Math. Gen. 35(37), R81–R150 (2002)
Kabashima, Y., Saad, D.: The Belief in TAP. Neural Information Processing Systems 11 (1998)
Yedidia, J., Freeman, W., Weiss, Y.: Generalized Belief Propagation. Neural Information Processing Systems 13 (2000)
Welling, M., Teh, Y.W.: Belief Optimization for Binary Networks: A Stable Alternative to Loopy Belief Propagation. In: 17th conference on Uncertainty in Artificial Intelligence, pp. 554–561 (2001)
Tonosaki, Y., Kabashima, Y.: Sequential Minimization of the Bethe Free Energy of Ising Spin Systems. Interdisciplinary Information Sciences 13(1), 57–64 (2007)
Kikuchi, R.: A Theory of Cooperative Phenomena. Phys. Rev. 81(6), 988–1003 (1951)
Yuille, A.L.: CCCP Algorithms to Minimize the Bethe and Kikuchi Free Energies: Convergent Alternatives to Belief Propagation. Neural Computation 14(7), 1691–1722 (2002)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nishiyama, Y., Watanabe, S. (2008). Generalization of Concave and Convex Decomposition in Kikuchi Free Energy. In: Kůrková, V., Neruda, R., Koutník, J. (eds) Artificial Neural Networks - ICANN 2008. ICANN 2008. Lecture Notes in Computer Science, vol 5163. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-87536-9_6
Download citation
DOI: https://doi.org/10.1007/978-3-540-87536-9_6
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-87535-2
Online ISBN: 978-3-540-87536-9
eBook Packages: Computer ScienceComputer Science (R0)