Abstract
In recent years, variational Bayesian learning has been used as an approximation of Bayesian learning. In spite of the computational tractability and good generalization performance in many applications, its statistical properties have yet to be clarified. In this paper, we analyze the statistical property in variational Bayesian learning of Bayesian networks which are widely used in information processing and uncertain artificial intelligence. We derive upper bounds for asymptotic variational stochastic complexities of Bayesian networks. Our result theoretically supports the effectiveness of variational Bayesian learning as an approximation of Bayesian learning.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Attias, H.: Inferring parameters and structure of latent variable models by variational Bayes. In: Proc. of UAI 1999, Stockholm, Sweden, pp. 21–30 (1999)
Beal, M.J.: Variational algorithms for approximate Bayesian inference, Ph.D. Thesis, University College London (2003)
Hinton, G., van Camp, D.: Keeping neural networks simple by minimizing the description length of weights. In: Proc. of COLT 1993, Santa Cruz, USA, pp. 5–13 (1993)
Hosino, T., Watanabe, S.: Stochastic complexity of variational Bayesian hidden Markov models. In: Proc. of IJCNN 2005, vol. 2, pp. 1114–1119 (2005)
Jensen, F.V.: Bayesian Networks and Decision Graphs. Springer, New York (2001)
Jordan, M.I.: Learning in graphical models. MIT Press, Cambridge (1999)
Mackay, D.J.: Bayesian interpolation. Neural Computation 4(2), 415–447 (1992)
Schwarz, G.: Estimating the dimension of a model. Annals of Statistics 6(2), 461–464 (1978)
Watanabe, K., Watanabe, S.: Variational Bayesian stochastic complexity of mixture models. In: Advances in NIPS, 18th edn., pp. 1465–1472. MIT Press, Cambridge (2006)
Watanabe, S.: Algebraic analysis for non-identifiable learning machines. Neural Computation 13(4), 899–933 (2001)
Whiley, M., Titterington, D.M.: Model identifiability in naive Bayesian networks, Technical Report 02-1, Department of Statistics, University of Glasgow (2002)
Yamazaki, K., Watanabe, S.: Stochastic complexity of Bayesian networks. In: Proc. of UAI 2003, Acapulco, Mexico, pp. 592–599 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2006 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Watanabe, K., Shiga, M., Watanabe, S. (2006). Upper Bounds for Variational Stochastic Complexities of Bayesian Networks. In: Corchado, E., Yin, H., Botti, V., Fyfe, C. (eds) Intelligent Data Engineering and Automated Learning – IDEAL 2006. IDEAL 2006. Lecture Notes in Computer Science, vol 4224. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11875581_17
Download citation
DOI: https://doi.org/10.1007/11875581_17
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-45485-4
Online ISBN: 978-3-540-45487-8
eBook Packages: Computer ScienceComputer Science (R0)