Abstract
We present a framework for approximate inference in probabilistic data models which is based on free energies. The free energy is constructed from two approximating distributions which encode different aspects of the intractable model. Consistency between distributions is required on a chosen set of moments. We find good performance using sets of moments which either specify factorized nodes or a spanning tree on the nodes.
The abstract should summarize the contents of the paper using at least 70 and at most 150 words. It will be set in 9-point font size and be inset 1.0 cm from the right and left margins. There will be two blank lines before and after the Abstract. ...
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Attias, H.: A variational Bayesian framework for graphical models. In: Leen, T., et al. (eds.) Advances in Neural Information Processing Systems, vol. 12, MIT Press, Cambridge (2000)
Bishop, C.M., Spiegelhalter, D., Winn, J.: VIBES: A variational inference engine for Bayesian networks. Advances in Neural Information Processing Systems 15 (2002)
Cornford, D., Csató, L., Evans, D.J., Opper, M.: Bayesian analysis of the scatterometer wind retrieval inverse problem: Some new approaches. Journal Royal Statistical Society B 66, 1–17 (2004)
Fabricius, T., Winther, O.: Correcting the bias of subtractive interference cancellation in cdma: Advanced mean field theory. Submitted to IEEE trans. Inf. Theory (2004)
Heskes, T., Albers, K., Kappen, H.: Approximate inference and constrained optimization. In: Proceedings of the 19th Annual Conference on Uncertainty in Artificial Intelligence (UAI 2003), San Francisco, CA, pp. 313–320. Morgan Kaufmann Publishers, San Francisco (2003)
Hojen-Sorensen, P.A.d.F.R., Winther, O., Hansen, L.K.: Mean field approaches to independent component analysis. Neural Computation 14, 889–918 (2002)
Malzahn, D., Opper, M.: An approximate analytical approach to resampling averages. Journal of Machine Learning Research 4, 1151–1173 (2003)
Malzahn, D., Opper, M.: Approximate analytical bootstrap averages for support vector classifiers. In: Proceedings of NIPS 2003 (2004)
Minka, T., Qi, Y.: Tree-structured approximations by expectation propagation. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, Cambridge, MA, vol. 16, MIT Press, Cambridge (2004)
Minka, T.P.: Expectation propagation for approximate bayesian inference. In: UAI 2001, pp. 362–369 (2001)
Minka, T.P.: A family of algorithms for approximate Bayesian inference. PhD thesis, MIT Media Lab (2001)
nonero Candela, Q., Winther, O.: Incremental gaussian processes. Advances in Neural Information Processing Systems 15, 1001–1008 (2003)
Opper, M., Saad, D.: Advanced Mean Field Methods: Theory and Practice. MIT Press, Cambridge (2001)
Opper, M., Winther, O.: Gaussian processes for classification: Mean field algorithms. Neural Computation 12, 2655–2684 (2000)
Opper, M., Winther, O.: Adaptive and self-averaging Thouless-Anderson- Palmer mean field theory for probabilistic modeling. Physical Review EÂ 64(056131) (2001)
Opper, M., Winther, O.: Tractable approximations for probabilistic models: The adaptive Thouless-Anderson-Palmer mean field approach. Physical Review 64(056131) (2001)
Opper, M., Winther, O.: Mean field methods for classification with gaussian processes. Advances in Neural Information Processing Systems 11, 309–315 (1999)
Plefka, T.: Convergence condition of the tap equation for the infinite-range ising spin glass. J. Phys. AÂ 15, 1971,1982
Wainwright, M., Jordan, M.I.: Semidefinite relaxations for approximate inference on graphs with cycles. In: Thrun, S., Saul, L., Schölkopf, B. (eds.) Advances in Neural Information Processing Systems, vol. 16, MIT Press, Cambridge (2004)
Wainwright, M.J., Jordan, M.I.: Semidefinite methods for approximate inference on graphs with cycles. Technical Report UCB/CSD-03-1226, UC Berkeley CS Division (2003)
Yedidia, J.S., Freeman, W.T., Weiss, Y.: Generalized belief propagation 13. In: Leen, T.K., Dietterich, T.G., Tresp, V. (eds.) NIPS, pp. 689–695 (2001)
Yuille, A.L.: Cccp algorithms to minimize the bethe and kikuchi free energies: convergent alternatives to belief propagation. Neural Comput. 14(7), 1691–1722 (2002)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Opper, M., Winther, O. (2004). Approximate Inference in Probabilistic Models. In: Ben-David, S., Case, J., Maruoka, A. (eds) Algorithmic Learning Theory. ALT 2004. Lecture Notes in Computer Science(), vol 3244. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30215-5_37
Download citation
DOI: https://doi.org/10.1007/978-3-540-30215-5_37
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23356-5
Online ISBN: 978-3-540-30215-5
eBook Packages: Springer Book Archive