Abstract
Aimed at the problems of small gradient, low learning rate, slow convergence error when the DBN using back-propagation process to fix the network connection weight and bias, proposing a new algorithm that combines with multi-innovation theory to improve standard DBN algorithm, that is the multi-innovation DBN(MI-DBN). It sets up a new model of back-propagation process in DBN algorithm, making the use of single innovation in previous algorithm extend to the use of innovation of the preceding multiple period, thus increasing convergence rate of error largely. To study the application of the algorithm in the social computing, and recognize the meaningful information about the handwritten numbers in social networking images. This paper compares MI-DBN algorithm with other representative classifiers through experiments. The result shows that MI-DBN algorithm, comparing with other representative classifiers, has a faster convergence rate and a smaller error for MNIST dataset recognition. And handwritten numbers on the image also have a precise degree of recognition.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Choi, B.: Multiagent social computing. Int. J. Web Portals 3(4), 56–68 (2011)
Lecun, Y., Bengio, Y., Hinton, G.E., et al.: Deep learning. Nature 521, 436–444 (2015)
Hinton, G.E., Osindero, S., Teh, Y.W.: A fast learning algorithm for deep belief nets. Neural Comput. 18(7), 1527–1554 (2006)
Larochelle, H., Erhan, D., Courville, A., et al.: An empirical evaluation of deep architectures on problems with many factors of variation. In: ICML 2007: 2007 International Conference on Machine Learning, pp. 473–480 (2007)
Keyvanrad, M.A., Homayounpour, M.M.: Deep belief network training improvement using elite samples minimizing free energy. Int. J. Pattern Recogn. Artif. Intell. 29(5), 1411–4046 (2014)
Liu, Y., Zhou, S., Chen, Q.: Discriminative deep belief networks for visual data classification. Pattern Recogn. 44(10–11), 2287–2296 (2011)
Hinton, G.E., Salakhutdinov, R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2016)
Ding, F., Tao, X., Tao, D.: Multi-innovation stochastic gradient identification methods. Control Theory Appl. 20(6), 870–874 (2003)
Ding, F.: Several multi-innovation identification methods. Digital Sig. Process. 20(4), 1027–1039 (2010)
Hinton, G.E.: A practical guide to training restricted boltzmann machines. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade, 2nd edn. LNCS, vol. 7700, pp. 599–619. Springer, Heidelberg (2012)
Swersky, K., Chen, B., Marlin, B., et al.: A tutorial on stochastic approximation algorithms for training Restricted Boltzmann Machines and Deep Belief Nets. In: Information Theory and Applications Workshop (ITA), pp. 1–10. IEEE (2010)
Schölkopf, B., Platt, J., Hofmann, T.: Greedy layer-wise training of deep networks. Adv. Neural Inform. Process. Syst. 19, 153–160 (2007)
Sarikaya, R., Hinton, G.E., Deoras, A.: Application of deep belief networks for natural language understanding. IEEE/ACM Trans. Audio Speech Lang. Process. 22(4), 778–784 (2014)
Ding, J., Xie, L., Ding, F.: Performance analysis of multi-innovation stochastic gradient identification for non- uniformly sampled systems. Control Decis. 26(9), 1338–1342 (2011)
Ding, F.: Multi-innovation identification theory and methods. J. Nanjing Univ. Inform. Sci. Technol. 4(1), 1–28 (2012)
Ding, F.: Convergence of multi innovation identification under attenuating excitation conditions for deterministic systems. J. Tsinghua Univ. 9, 111–115 (1998)
Lee, H., Ekanadham, C., Ng, A.: Sparse deep belief net model for visual area V2. Adv. Neural Inform. Process. Syst. 20, 873–880 (2008)
Krizhevsky, A.: Learning multiple layers of features from tiny images. University of Toronto, Canada, 17 (2009)
Salakhutdinov, R., Hinton, G.E.: Using deep belief nets to learn covariance kernels for gaussian processes. Adv. Neural Inform. Process. Syst. 20, 1249–1256 (2007)
Lecun, Y., Cortes, C.: The MNIST database of handwritten digits [DB/OL] (2011). http://yann.lecun.com/exdb/mnist/index.html
Keyvanrad, M.A., Homayounpour, M.M.: A brief survey on deep belief networks and introducing a new object oriented MATLAB toolbox (DeeBNetV2.2). Comput. Vis. Pattern Recogn. 12, 1408–3264 (2014)
Tieleman, T., Hinton, G.: Using fast weights to improve persistent contrastive divergence. In: ICML 2009: Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Quebec, Canada, pp. 1033–1040 (2009)
Acknowledgment
This work is partially supported by Shanxi Nature Foundation (No.2015011045). The authors also gratefully acknowledge the helpful comments and suggestions of the reviewers, which have improved the presentation.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer Science+Business Media Singapore
About this paper
Cite this paper
Qin, P., Li, M., Miao, Q., Li, C. (2016). Research of the DBN Algorithm Based on Multi-innovation Theory and Application of Social Computing. In: Che, W., et al. Social Computing. ICYCSEE 2016. Communications in Computer and Information Science, vol 623. Springer, Singapore. https://doi.org/10.1007/978-981-10-2053-7_51
Download citation
DOI: https://doi.org/10.1007/978-981-10-2053-7_51
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-2052-0
Online ISBN: 978-981-10-2053-7
eBook Packages: Computer ScienceComputer Science (R0)