Skip to main content

Advertisement

Springer Nature Link
Log in

Discriminative matrix-variate restricted Boltzmann machine classification model

  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

Matrix-variate Restricted Boltzmann Machine (MVRBM), a variant of Restricted Boltzmann Machine, has demonstrated excellent capacity of modelling matrix variable. However, MVRBM is still an unsupervised generative model, and is usually used to feature extraction or initialization of deep neural network. When MVRBM is used to classify, additional classifiers must be added. In order to make the MVRBM itself be supervised, in this paper, we propose improved MVRBMs for classification, which can be used to classify 2D data directly and accurately. To this end, on one hand, classification constraint is added to MVRBM to get Matrix-variate Restricted Boltzmann Machine Classification Model (ClassMVRBM). On the other hand, fisher discriminant analysis criterion for matrix-style variable is proposed and applied to the hidden variable, therefore, the extracted feature is more discriminative so as to enhance the classification performance of ClassMVRBM. We call the novel model Matrix-variate Restricted Boltzmann Machine Classification Model with Fisher discriminant analysis (ClassMVRBM-MVFDA). Experimental results on some publicly available databases demonstrate the superiority of the proposed models. Of which, the image classification accuracy of ClassMVRBM is higher than conventional unsupervised RBM, its variants and supervised Restricted Boltzmann Machine Classification Model (ClassRBM) for vector variable. Especially, the image classification accuracy of the proposed ClassMVRBM-MVFDA performs better than supervised ClassMVRBM and vectorial RBM-FDA.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Chen, L., Jiang, D., Song, H., et al. (2018). A lightweight end-side user experience data collection system for quality evaluation of multimedia communications. IEEE Access, 2018(6), 15408–15419.

    Article  Google Scholar 

  2. Sun, M., Jiang, D., Song, H., et al. (2017). Statistical resolution limit analysis of two closely-spaced signal sources using Rao test. IEEE Access, 5, 22013–22022.

    Article  Google Scholar 

  3. Wang, H., & Wang, J. (2013). 2DPCA with L1-norm for simultaneously robust and sparse modelling. Neural Network, 46(10), 190–198.

    Article  Google Scholar 

  4. Yang, J., Zhang, D., Frangi, A., & Yang, J. (2004). Two-dimensional PCA: A new approach to appearance-based face representation and recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(1), 131–137.

    Article  Google Scholar 

  5. Ju, F., Sun, Y., Gao, J., Hu, Y., & Yin, B. (2015). Image outlier detection and feature extraction via L1-norm-based 2D probabilistic PCA. IEEE Transactions on Image Processing, 24(12), 4834–4846.

    Article  MathSciNet  Google Scholar 

  6. Li, M., & Yuan, B. (2005). 2D-LDA: A statistical linear discriminant analysis for image matrix. Pattern Recognition Letters, 26(5), 527–532.

    Article  Google Scholar 

  7. Fischer, A., & Igel, C. (2012). An introduction to restricted Boltzmann machines. In Progress in pattern recognition, image analysis, computer vision, and applications (pp. 14–36). Springer.

  8. Hinton, G. E., & Salakhutdinov, R. (2006). Reducing the dimensionality of data with neural networks. Science, 313(5786), 504–507.

    Article  MathSciNet  Google Scholar 

  9. Wang, J., Wang, W., Wang, R., & Gao, W. (2015). Image classification using RBM to encode local descriptors with group sparse learning. In Proceedings of international conference on image processing (pp. 912–916). Canada: IEEE

  10. Vinod Nair, G. E. H. (2010). Rectified linear units improve restricted Boltzmann machines. In Proceedings of the 27th international conference on machine learning (pp. 807–814). Israel.

  11. Cho, K., Ilin, A., & Raiko, T. (2011) Improved learning of Gaussian-Bernoulli restricted Boltzmann machines. In Proceedings of the twenty-first international conference on artificial neural networks (pp. 10–17). Springer.

  12. Nguyen, T., Tran, T., Phung, D., & Venkatesh, S. (2015). Tensor-variate restricted Boltzmann machines. In Proceedings of the twenty-ninth national conference on artificial intelligence (pp. 2887–2893). USA: AAAI.

  13. Salakhutdinov, R., Mnih, A., & Hinton, G. (2007). Restricted Boltzmann machines for collaborative filtering. In Proceedings of the twenty-fourth international conference on machine learning (pp. 791–798). USA: ACM.

  14. Dahl, G. E., Dong, Y., Li, D., & Acero, A. (2011). Context-dependent pre-trained deep neural networks for large-vocabulary speech recognition. IEEE Transactions on Audio, Speech and Language Processing, 20(1), 30–42.

    Article  Google Scholar 

  15. Larochelle, H., Mandel, M., Pascanu, R., et al. (2012). Learning algorithms for the classification restricted Boltzmann machine. Journal of Machine Learning Research, 13(1), 643–669.

    MathSciNet  MATH  Google Scholar 

  16. Peng, X., Gao, X., & Li, X. (2017). An infinite classification RBM model for radar HRRP recognition. In International joint conference on neural networks (pp. 1442–1448). USA: IEEE.

  17. Qi, G., Sun, Y., Gao, J., Hu, Y., & Li, J. (2016). Matrix variate restricted boltzmann machine. In The proceeding of 2016 international joint conference on neural networks (pp. 389–395). Canada: IEEE.

  18. Liu, S., Sun, Y., Hu, Y., Gao, J., Ju, F., & Yin, B. (2017). Matrix variate RBM model with gaussian distributions. In The proceeding of 2017 international joint conference on neural networks (pp. 808–815). USA: IEEE.

  19. Jiang, D., Wang, W., Shi, L., et al. (2018). A compressive sensing-based approach to end-to-end network traffic reconstruction. IEEE Transactions on Network Science and Engineering, 5(3), 1–12.

    Google Scholar 

  20. Wang, F., Jiang, D., Wen, H., et al. (2019). Adaboost-based security level classification of mobile intelligent terminals. The Journal of Supercomputing 1–19.

  21. Xie, G. S., Zhang, X. Y., Zhang, Y. M., et al. (2014). Integrating supervised subspace criteria with restricted Boltzmann machine for feature extraction. In International joint conference on neural networks (pp. 1622–1629). IEEE.

  22. Jiang, D., Wang, Y., Lv, Z., et al. (2019). Big data analysis-based network behavior insight of cellular networks for industry 4.0 applications. IEEE Transactions on Industrial Informatics., 10, 15–20. https://doi.org/10.1109/tii.2019.2930226.

    Article  Google Scholar 

  23. Jiang, D., Huo, L., Lv, Z., et al. (2018). A joint multi-criteria utility-based network selection approach for vehicle-to-infrastructure networking. IEEE Transactions on Intelligent Transportation Systems, 19(10), 3305–3319.

    Article  Google Scholar 

  24. Jiang, D., Huo, L., & Song, H. (2018). Rethinking behaviors and activities of base stations in mobile cellular networks based on big data analysis. IEEE Transactions on Network Science and Engineering, 1(1), 1–12.

    MathSciNet  Google Scholar 

  25. Jiang, D., Zhang, P., Lv, Z., et al. (2016). Energy-efficient multi-constraint routing algorithm with load balancing for smart city applications. IEEE Internet of Things Journal, 3(6), 1437–1447.

    Article  Google Scholar 

  26. Zhu, J., Song, Y., Jiang, D., et al. (2018). A new deep-Q-learning-based transmission scheduling mechanism for the cognitive Internet of things. IEEE Internet of Things Journal, 5(4), 2375–2385.

    Article  Google Scholar 

  27. Gao, J., Guo, Y., & Wang, Z. (2017). Matrix neural networks. In International symposium on neural networks (pp. 313–320). Springer.

  28. Wong, W. K., & Sun, M. (2011). Deep learning regularized fisher mappings. IEEE Transactions on Neural Networks, 22(10), 1668–1675.

    Article  Google Scholar 

  29. Yang, J., & Yang, J. Y. (2002). From Image vector to matrix: A straightforward image projection technique—IMPCA vs PCA. Pattern Recognition, 35(9), 1997–1999.

    Article  Google Scholar 

  30. LeCun, Y., Bottou, L., Bengio, Y., & Haffner, P. (1998). Gradient-based learning applied to document recognition. In Proceedings of the IEEE (Vol. 86, No. (11), pp. 2278–2324). IEEE.

  31. Wang, Y., & Mori, G. (2009). Human action recognition by semilatent topic models. IEEE Transactions on Pattern Analysis & Machine Intelligence, 31(10), 1762.

    Article  Google Scholar 

  32. Leibe, B., & Schiele, B. (2003). Analyzing appearance and contour based methods for object categorization. In Proceedings of IEEE computer society conference on computer vision and pattern recognition (pp. 1–7). USA: IEEE.

  33. Nene, S., Nayar, S., & Murase, H. (1996). Columbia object image library (COIL-20). Technical report CUCS-005-96, USA.

  34. Hinton, G. E. (2012). A practical guide to training restricted Boltzmann machine. LNCS, 7700, 599–619.

    Google Scholar 

  35. Tenenbaum, J., Silva, V., & Langford, J. (2000). A global geometric framework for nonlinear dimensionality reduction. Science, 290(5500), 2319–2323.

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the Natural Science Foundation of China (Nos. 61402024, 61772049, 61632006, 61876012), and Scientific Research Common Program of Beijing Municipal Commission of Education (Nos. KM201710005022, KM201510005024) and Beijing Key Laboratory of Computational Intelligence and Intelligent System.

Author information

Authors and Affiliations

  1. Beijing Key Laboratory of Multimedia and Intelligent Software Technology, Faculty of Information Technology, Beijing University of Technology, Beijing, 100124, China

    Jinghua Li, Pengyu Tian, Dehui Kong, Lichun Wang & Shaofan Wang

  2. College of Computer Science and Technology, Faculty of Electronic Information and Electrical Engineering, Dalian University of Technology, Dalian, 116620, China

    Baocai Yin

Authors
  1. Jinghua Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pengyu Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dehui Kong
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lichun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shaofan Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Baocai Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinghua Li.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Tian, P., Kong, D. et al. Discriminative matrix-variate restricted Boltzmann machine classification model. Wireless Netw 27, 3621–3633 (2021). https://doi.org/10.1007/s11276-019-02234-w

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-019-02234-w

Keywords