Abstract
In this paper, which is based on the important recent work of Yedidia, Freeman, and Weiss, we present a generalized form of belief propagation, viz. belief propagation on a partially ordered set (PBP). PBP is an iterative message-passing algorithm for solving, either exactly or approximately, the marginalized product density problem, which is a general computational problem of wide applicability. We will show that PBP can be thought of as an algorithm for minimizing a certain “free energy” function, and by exploiting this interpretation, we will exhibit a one-to-one correspondence between the fixed points of PBP and the stationary points of the free energy.
This research was supported by NSF grant no. CCR-0118670, and grants from Sony, Qualcomm. and Caltech’s Lee Center for Advanced Networking.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
S.M. Aji and R.J. McEliece, “The generalized distributive law,” IEEE Trans. Inform. Theory, 46(2): 325–343 (March 2000).
S.M. Aji and R.J. McEliece, “The generalized distributive law and free energy minimization,” Proc. 2001 Allerton Conf. Comm. Control and Computing (Oct. 2001).
T.M. Cover and J. A. Thomas, Elements of Information Theory. New York: John Wiley and Sons, 1991.
W.H. Fleming, Functions of Several Variables. Reading, Mass.: Addison-Wesley, 1965.
R.A. Horn and C.R. Johnson, Matrix Analysis. Cambridge: Cambridge University Press, 1985.
F.R. Kschichang, B.J. Frey, and H.-A. Loeliger, “Factor graphs and the sum-product algorithm,” IEEE Trans. Inform. Theory, 47(2): 498–519 (Feb. 2001).
P. Pakzad and V. Anantharam, “Belief propagation and statistical physics,” Proc. 2002 Conf. Inform. Sciences and Systems, Princeton U., March 2002.
J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Francisco: Morgan Kaufmann, 1988.
G.R. Shafer and P.P. Shenoy, “Probability propagation,” Ann. Mat. Art. Intell., 2: 327–352 (1990).
R.P. Stanley, Enumerative Combinatorics, Vol. 1. (Cambridge Studies in Advanced Mathematics 49) Cambridge: Cambridge University Press, 1997.
J.S. Yedidia, “An idiosyncratic journey beyond mean field theory,” pp. 21–35 in Advanced Mean Field Methods, Theory and Practice, eds. Manfred Opper and David Saad, MIT Press, 2001.
J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Generalized belief propagation,” pp. 689–695 in Advances in Neural Information Processing Systems 13 (2000) eds. Todd K. Leen, Thomas G. Dietterich, and Volker Tresp.
J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Bethe free energy, Kikuchi approximations, and belief propagation algorithms,” available at http://www.merl.com/papers/TR2001-16/
J.S. Yedidia, W.T. Freeman, and Y. Weiss, “Constructing free energy approximations and generalized belief propagation algorithms,” available at http://www.merl.com/papers/TR2002-35/
J.M. Yeomans, Statistical Mechanics of Phase Transitions. Oxford: Oxford University Press, 1992.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag New York, Inc.
About this paper
Cite this paper
McEliece, R.J., Yildirim, M. (2003). Belief Propagation on Partially Ordered Sets. In: Rosenthal, J., Gilliam, D.S. (eds) Mathematical Systems Theory in Biology, Communications, Computation, and Finance. The IMA Volumes in Mathematics and its Applications, vol 134. Springer, New York, NY. https://doi.org/10.1007/978-0-387-21696-6_10
Download citation
DOI: https://doi.org/10.1007/978-0-387-21696-6_10
Publisher Name: Springer, New York, NY
Print ISBN: 978-1-4419-2326-4
Online ISBN: 978-0-387-21696-6
eBook Packages: Springer Book Archive