Abstract
In this paper, we present an efficient learning algorithm for Deep Boltzmann Machine (DBM) to get the data-dependent expectation quickly. The algorithm adopts a layer-wise accelerating inference strategy to compute the mean values of all hidden layers, instead of the mean values by repeatedly running the equations of mean-field fixed-point until convergence. By taking advantage of layer-wise inference strategy, we can rapidly get the approximate mean values in a few iterations. This strategy also could learn efficiently a high performance model for high-dimensional high-structured sensory inputs. The proposed algorithm with layer-wise accelerating inference performs well compared to original DBM with given learning tasks.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Computation 18(7), 1527–1554 (2006)
Hinton, G.E.: A practical guide to training restricted boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade, 2nd edn. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012)
Salakhutdinov, R., Hinton, G.E.: A Better Way to Pretrain Deep Boltzmann Machines. NIPS 3, 2456–2464 (2012)
Salakhutdinov, R., Hinton, G.: An efficient learning procedure for deep Boltzmann machines. Neural Computation 24(8), 1967–2006 (2012)
Salakhutdinov, R., Larochelle, H.: Efficient learning of deep Boltzmann machines. In: International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 693–700 (2010)
Nair V, Hinton G E.: Implicit mixtures of restricted Boltzmann machines. Advances in Neural Information Processing Systems, pp. 1145-1152 (2009)
Salakhutdinov, R., Hinton, G.E.: Deep boltzmann machines. In: International Conference on Artificial Intelligence and Statistics, vol. 9, pp. 448–455 (2009)
Nair, V., Hinton, G.E.: 3D object recognition with deep belief nets. In: Advances in Neural Information Processing Systems, pp. 1339–1347 (2009)
Salakhutdinov, R.: Learning deep Boltzmann machines using adaptive MCMC. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 943–950 (2010)
Erhan, D., Bengio, Y., Courville, A., et al.: Why does unsupervised pre-training help deep learning? The Journal of Machine Learning Research 11, 625–660 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Wang, J., Zhang, X. (2014). Efficient Deep Learning Algorithm with Accelerating Inference Strategy. In: Luo, X., Yu, J.X., Li, Z. (eds) Advanced Data Mining and Applications. ADMA 2014. Lecture Notes in Computer Science(), vol 8933. Springer, Cham. https://doi.org/10.1007/978-3-319-14717-8_31
Download citation
DOI: https://doi.org/10.1007/978-3-319-14717-8_31
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-14716-1
Online ISBN: 978-3-319-14717-8
eBook Packages: Computer ScienceComputer Science (R0)