Abstract
In hierarchical learning machines such as neural networks, Bayesian learning provides better generalization performance than maximum likelihood estimation. However, its accurate approximation using a Markov chain Monte Carlo (MCMC) method requires huge computational cost. The exchange Monte Carlo (EMC) method was proposed as an improved algorithm of MCMC method. Although its effectiveness has been shown not only in Bayesian learning but also in many fields, the mathematical foundation of EMC method has not yet been established. In our previous work, we analytically clarified the asymptotic behavior of average exchange ratio, which is used as a criterion for designing the EMC method. In this paper, we verify the accuracy of our result by comparing the theoretical value of average exchange ratio with the experimental value, and propose the method to check the convergence of EMC method based on our theoretical result.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Aoyagi, M., Watanabe, S.: Stochastic complexities of reduced rank regression in Bayesian estimation. Neural Networks 18(7), 924–933 (2005)
Atiyah, M.F.: Resolution of singularities and division of distributions. Communications of Pure and Applied Mathematics 13, 145–150 (1970)
Berg, B.A., Neuhaus, T.: Multicanonical algorithms for first order phase transitions. Physics Letter B 267(2), 249–253 (1991)
Hukushima, K.: Domain Wall Free Energy of Spin Glass Models: Numerical Method and Boundary Conditions. Physical Review E 60, 3606–3613 (1999)
Hukushima, K., Nemoto, K.: Exchange Monte Carlo Method and Application to Spin Glass Simulations. Journal of Physical Society of Japan 65(6), 1604–1608 (1996)
Hukushima, K.: Extended ensemble Monte Carlo approach to hardly relaxing problems. Computer Physics Communications 147, 77–82 (2002)
Iba, Y.: Extended Ensemble Monte Carlo. International Journal of Modern Physics C 12, 623–656 (2001)
Liang, F.: An effective Bayesian neural network classifier with a comparison study to support vector machine. Neural Computation 15, 1959–1989 (2003)
Marinari, E., Parisi, G.: Simulated tempering: a new Monte Carlo scheme. Europhysics Letters 19(6), 451–455 (1992)
Nagata, K., Watanabe, S.: Analysis of Exchange Ratio for Exchange Monte Carlo Method. In: Proc. of the First IEEE Symposium on Foundation of Computational Intelligence (FOCI 2007), pp. 434–439 (2007)
Pinn, K., Wieczerkowski, C.: Number of Magic Squares from Parallel Tempering Monte Carlo. International Journal of modern Physics 9(4), 541–546 (1998)
Sugita, Y., Okamoto, Y.: Replica-exchange molecular dynamics method for protein folding. Chemical Physics Letters 314(1-2), 141–151 (1999)
Watanabe, S.: Algebraic Analysis for Nonidentifiable Learning Machines. Neural Computation 13, 899–933 (2001)
Yamazaki, K., Watanabe, S.: Singularities in mixture models and upper bounds of stochastic complexity. Neural networks 16(7), 1029–1038 (2003)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nagata, K., Watanabe, S. (2008). Experimental Analysis of Exchange Ratio in Exchange Monte Carlo Method. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4985. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69162-4_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-69162-4_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69159-4
Online ISBN: 978-3-540-69162-4
eBook Packages: Computer ScienceComputer Science (R0)