Abstract
Restricted Boltzmann machines (RBMs) are widely applied to solve many machine learning problems. Usually, the cost function of RBM is log-likelihood function of marginal distribution of input data, and the training method involves maximizing the cost function. Distribution of the trained RBM is identical to that of input data. But the reconstruction error always exists even the distributions are almost identical. In this paper, a method to train RBM by adding reconstruction error to the cost function is put forward. Two categories of trials are performed to validate the proposed method: feature extraction and classification. The experimental results show that the proposed method can be effective.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fischer, A., Igel, C.: Training restricted Boltzmann machines: an introduction. Pattern Recogn. 47(1), 25–39 (2014)
Salakhutdinov, R., Mnih, A., Hinton, G.: Restricted Boltzmann machines for collaborative filtering. In: International Conference on Machine Learning, vol. 227, pp. 791–798. ACM (2007)
Xie, G.S., Zhang, X.Y., Zhang, Y.M., Liu, C.L.: Integrating supervised subspace criteria with restricted Boltzmann machine for feature extraction. In: International Joint Conference on Neural Networks, pp. 1622–1629. IEEE Press, New York (2014)
Zhang, K., Liu, J., Chai, Y., Qian, K.: An optimized dimensionality reduction model for high-dimensional data based on restricted Boltzmann machines. In: Control and Decision Conference, pp. 2939–2944. IEEE Press, New York (2015)
Salakhutdinov, R., Tenenbaum, J.B., Torralba, A.: Learning with hierarchical-deep models. IEEE Trans. Pattern Anal. Mach. Intell. 35(8), 1958–1971 (2013)
Larochelle, H., Mandel, M., Pascanu, R., Bengio, Y.: Learning algorithms for the classification restricted Boltzmann machine. J. Mach. Learn. Res. 13(1), 643–669 (2012)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Hinton, G.E.: To recognize shapes, first learn to generate images. Prog. Brain Res. 165, 535–547 (2007)
Ji, N.N., Zhang, J.S., Zhang, C.X.: A sparse-response deep belief network based on rate distortion theory. Pattern Recogn. 47(9), 3179–3191 (2014)
Srivastava, N., Salakhutdinov, R.: Multimodal learning with deep Boltzmann machines. In: International Conference on Neural Information Processing Systems, pp. 2222–2230. IEEE Press, New York (2012)
Luo, L., Wang, Y., Peng, H., Tang, Z., You, S., Huang, X.: Training restricted Boltzmann machine with dynamic learning rate. In: International Conference on Computer Science & Education. IEEE (2016)
Yu, J., Gwak, J., Lee, S., Jeon, M.: An incremental learning approach for restricted Boltzmann machines. In: International Conference on Control, Automation and Information Sciences, pp. 113–117. IEEE Press, New York (2015)
Golovko, V., Kroshchanka, A., Turchenko, V., Jankowski, S., Treadwell, D.: A new technique for restricted Boltzmann machine learning. In: International Conference on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications, vol. 1, pp. 182–186. IEEE Press, New York (2015)
Huang, W., Hong, H., Bian, K., Zhou, X., Song, G., Xie, K.: Improving deep neural network ensembles using reconstruction error. In: 2015 International Joint Conference on Neural Networks (IJCNN), pp. 1–7. IEEE Press, New York (2015)
Yin, J., Lv, J., Sang, Y., Guo, J.: Classification model of restricted Boltzmann machine based on reconstruction error. Neural Comput. Appl. 1–16 (2016)
Hinton, G.E.: Training products of experts by minimizing contrastive divergence. Neural Comput. 14(8), 1771–1800 (2002)
Acknowledgments
This work was Supported by National Natural Science Fund for Distinguished Young Scholar Grant No. 61625204).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Yin, J., Mao, Q., Liu, D., Xu, Y., Lv, J. (2018). Method to Improve the Performance of Restricted Boltzmann Machines. In: Huang, T., Lv, J., Sun, C., Tuzikov, A. (eds) Advances in Neural Networks – ISNN 2018. ISNN 2018. Lecture Notes in Computer Science(), vol 10878. Springer, Cham. https://doi.org/10.1007/978-3-319-92537-0_14
Download citation
DOI: https://doi.org/10.1007/978-3-319-92537-0_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-92536-3
Online ISBN: 978-3-319-92537-0
eBook Packages: Computer ScienceComputer Science (R0)