Abstract
In singular statistical models, it was shown that Bayes learning is effective. However, on Bayes learning, calculation containing the Bayes posterior distribution requires huge computational costs. To overcome the problem, mean field approximation (or equally variational Bayes method) was proposed. Recently, the generalization error and stochastic complexity in mean field approximation have been theoretically studied. In this paper, we treat the complete bipartite graph-type Boltzmann machines and derive the upper bound of the asymptotic stochastic complexity in mean field approximation.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Watanabe, S.: Algebraic analysis for nonidentifiable learning machines. Neural Computation 13(4), 899–933 (2001)
Aoyagi, M., Watanabe, S.: The generalization error of reduced rank regression in bayesian estimation. In: Proc. of ISITA 2004, Italy, pp. 1068–1073 (2004)
Yamazaki, K., Watanabe, S.: Singularities in mixture models and upper bounds of stochastic complexity. International Journal of Neural Networks 16(7), 1029–1038 (2003)
Nakajima, S., Watanabe, S.: Generalization Error and Free Energy of Linear Neural Networks in Variational Bayes Approach. In: Proc. of ICONI 2005, Taiwan, pp. 55–60 (2005)
Watanabe, K., Watanabe, S.: Lower bounds of stochastic complexities in variational Bayes learning of gaussian mixture models. In: Proc. IEEE conference on Cybernetics and Intelligent Systems, pp. 99–104 (2004)
Hosino, T., Watanabe, K., Watanabe, S.: Stochastic Complexity of Variational Bayesian Hidden Markov Models. In: Proc. of IJCNN 2005, Canada (2005)
Hosino, T., Watanabe, K., Watanabe, S.: Stochastic Complexity of Stochastic Context Free Grammer on Variational Bayesian method. IEICE Technicalreport, NC2005-49 (October 2005)
Nakano, N., Watanabe, S.: Stochastic Complexity of Layered Neural Networks in Mean Field Approximation. In: Proc. of ICONI 2005, Taiwan, pp. 332–337 (2005)
Nishiyama, Y., Watanabe, S.: Asymptotic Behavior of Free Energy of General Boltzmann Machines in Mean Field Approximation, IEICE Technical report, (July 2006) (to appear)
Yamazaki, K., Watanabe, S.: Singularities in complete bipartite graph-type boltzmann machines and upper bounds of stochastic complexities. IEEE Trans. Neural Networks 16(2), 312–324 (2005)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Nishiyama, Y., Watanabe, S. (2006). Asymptotic Behavior of Stochastic Complexity of Complete Bipartite Graph-Type Boltzmann Machines. In: King, I., Wang, J., Chan, LW., Wang, D. (eds) Neural Information Processing. ICONIP 2006. Lecture Notes in Computer Science, vol 4232. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11893028_47
Download citation
DOI: https://doi.org/10.1007/11893028_47
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-46479-2
Online ISBN: 978-3-540-46480-8
eBook Packages: Computer ScienceComputer Science (R0)