Skip to main content
SpringerLink
Log in

Analysis on Equilibrium Point of Expectation Propagation Using Information Geometry

  • Conference paper
Advances in Neuro-Information Processing (ICONIP 2008)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 5507))

Included in the following conference series:

  • 1577 Accesses

Abstract

Expectation Propagation (EP) extends belief propagation by approximating messages with expectations of statistics, in which users can choose the statistics. In this paper, we discuss how a choice of statistics affects accuracy of EP’s estimates. We approximate estimation error of EP via perturbation analysis based on information geometry. By comparing the approximated estimation error, we show that adding statistics does not necessarily improve the accuracy of EP. A numerical example confirms validity of our analytical results.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Minka, T.: A Family of Algorithms for Approximate Bayesian Inference. Ph.D. thesis, Massachusetts Institute of Technology (2001)

    Google Scholar 

  2. Heskes, T., Opper, M., Wiegerinck, W., Winther, O., Zoeter, O.: Approximate inference techniques with expectation constraints. J. Stat. Mech.: Theory Exp., P11015 (2005)

    Google Scholar 

  3. Wainwright, M.J.: Stochastic Processes on Graphs with Cycles. Ph.D. thesis, Massachusetts Institute of Technology (2002)

    Google Scholar 

  4. Amari, S., Nagaoka, H.: Methods of Information Geometry. Translations of Mathematical Monographs, vol. 191. American Mathematical Society (2001)

    Google Scholar 

  5. Ikeda, S., Tanaka, T., Amari, S.: Information geometry of turbo and low-density parity-check codes. IEEE Trans. Info. Theory 50, 1097–1114 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  6. Ikeda, S., Tanaka, T., Amari, S.: Stochastic reasoning, free energy, and information geometry. Neural Comput. 16, 1779–1810 (2004)

    Article  MATH  Google Scholar 

  7. Welling, M.: On the choice of regions for generalized belief propagation. In: Proc. 20th Conf. Uncertainty in AI, pp. 585–592 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Graduate School of Informatics, Kyoto University, Kyoto, 606-8501, Japan

    Hideyuki Matsui & Toshiyuki Tanaka

Authors
  1. Hideyuki Matsui
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Toshiyuki Tanaka
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Network Design and Research Center, Kyushu Institute of Technology,, 680-4, Kawazu, Iizuka,, 820-8502, Fukuoka, Japan

    Mario Köppen

  2. Knowledge Engineering and Discovery Research Institute (KEDRI), School of Computing and Mathematical Sciences, Auckland University of Technology, 350 Queen Street, 10110, Auckland, New Zealand

    Nikola Kasabov

  3. Department of Electrical and Computer Engineering, Robotics Laboratory, Auckland University of Technology, 38 Princes Street,, 1142, Auckland, New Zealand

    George Coghill

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Matsui, H., Tanaka, T. (2009). Analysis on Equilibrium Point of Expectation Propagation Using Information Geometry. In: Köppen, M., Kasabov, N., Coghill, G. (eds) Advances in Neuro-Information Processing. ICONIP 2008. Lecture Notes in Computer Science, vol 5507. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-03040-6_19

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-03040-6_19

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-03039-0

  • Online ISBN: 978-3-642-03040-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation