Skip to main content
SpringerLink
Log in

Ensemble RBM-based classifier using fuzzy integral for big data classification

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The restricted Boltzmann machine (RBM) is a primary building block of deep learning models. As an efficient representation learning approach, deep RBM can effectively extract sophisticated and informative features from raw data. Little research has been undertaken on using deep RBM to extract features from big data however. In this paper, we investigate this problem, and an ensemble approach for big data classification based on Hadoop MapReduce and fuzzy integral is proposed. The proposed method consists of two stages, map and reduce. In the map stage, multiple RBM-based classifiers used for ensemble are trained in parallel. In the reduce stage, the trained multiple RBM-based classifiers are integrated by fuzzy integral. Experiments on five big data sets show that the proposed approach can outperform other baseline methods to achieve state-of-the-art performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Bengio Y, Courville A, Vincent P (2013) Representation learning: a review and new perspectives. IEEE Trans Pattern Anal Mach Intell 35(8):1798–1828

    Article  Google Scholar 

  2. Zhong GQ, Wang LN, Ling X et al (2016) An overview on data representation learning: from traditional feature learning to recent deep learning. J Financ Data Sci 2(4):265–278

    Google Scholar 

  3. Ding SF, Zhang N, Zhang J et al (2017) Unsupervised extreme learning machine with representational features. Int J Mach Learn Cybern 8(2):587–595

    Google Scholar 

  4. Kasun LLC, Zhou H, Huang GB et al (2013) Representational learning with ELMs for big data. Intell Syst IEEE 28(6):31–34

    Google Scholar 

  5. Huang P, Qian CS, Yang G et al (2018) Local mean representation based classifier and its applications for data classification. Int J Mach Learn Cybern 9(6):969–978

    Google Scholar 

  6. Chen SG, Wu XJ (2017) Multiple birth least squares support vector machine for multi-class classification. Int J Mach Learn Cybern 8(6):1731–1742

    Google Scholar 

  7. Chen SS, Cao JJ, Gan LL et al (2018) Experimental study on generalization capability of extended naive Bayesian classifier. Int J Mach Learn Cybern 9(1):5–19

    Google Scholar 

  8. Huang GB, Zhou HM, Ding XJ et al (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybern) 42(2):513–529

    Google Scholar 

  9. Shen XJ, Dong Y, Gou JP et al (2018) Least squares kernel ensemble regression in reproducing kernel Hilbert space. Neurocomputing 311:235–244

    Google Scholar 

  10. Wang D, Wang P, Shi JZ (2018) A fast and efficient conformal regressor with regularized extreme learning machine. Neurocomputing 304:1–11

    Google Scholar 

  11. Borgi MA, Nguyen TP, Labate D et al (2018) Statistical binary patterns and post-competitive representation for pattern recognition. Int J Mach Learn Cybern 9(6):1023–1038

    Google Scholar 

  12. Chen YH, Tong SG, Cong FY et al (2016) Symmetrical singular value decomposition representation for pattern recognition. Neurocomputing 214:143–154

    Google Scholar 

  13. Yang ZJ, Huang P, Wan MH et al (2018) Local descriptor margin projections (LDMP) for face recognition. Int J Mach Learn Cybern 9(8):1387–1398

    Google Scholar 

  14. Pearson K (1901) On lines and planes of closest fit to systems of points in space. Philos Mag 2:559–572

    MATH  Google Scholar 

  15. Fisher R (1936) The use of multiple measurements in taxonomic problems. Ann Hum Genet 7:179–188

    Google Scholar 

  16. Roweis S, Saul L (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326

    Google Scholar 

  17. Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507

    MathSciNet  MATH  Google Scholar 

  18. Lecun Y, Bengio Y, Hinton GE (2015) Deep learning. Nature 521:436–444

    Google Scholar 

  19. Zhang QC, Yang LT, Chen ZK et al (2018) A survey on deep learning for big data. Inf Fusion 42:146–157

    Google Scholar 

  20. Zhang N, Ding SF, Zhang J et al (2018) An overview on restricted Boltzmann machines. Neurocomputing 275:1186–1199

    Google Scholar 

  21. Fischer A, Igel C (2014) Training restricted Boltzmann machines: an introduction. Pattern Recognit 47:25–39

    MATH  Google Scholar 

  22. Yu W, de la Rosa E (2018) Deep Boltzmann machine for nonlinear system modelling. Int J Mach Learn Cybern. https://doi.org/10.1007/s13042-018-0847-0

    Google Scholar 

  23. Wang XZ, Zhang TL, Wang R (2017) Non-iterative deep learning: incorporating restricted Boltzmann machine into multilayer random weight neural networks. IEEE Trans Syst Man Cybern Syst. https://doi.org/10.1109/TSMC.2017.2701419

    Google Scholar 

  24. Weng R, Lu J, Tan Y et al (2016) Learning cascaded deep auto-encoder networks for face alignment. IEEE Trans Multimed 18(10):2066–2078

    Google Scholar 

  25. Lecun Y, Bottou L, Bengio Y et al (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324

    Google Scholar 

  26. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1725–1780

    Google Scholar 

  27. Goodfellow I, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial nets. Adv Neural Inf Process Syst 1:2672–2680

    Google Scholar 

  28. Zhao SY, Wang XZ, Chen DG et al (2013) Nested structure in parameterized rough reduction. Inf Sci 248:130–150

    MathSciNet  MATH  Google Scholar 

  29. Hinton G (1999). Products of experts. In: Proceedings of the ninth international conference on artificial neural networks (ICANN), Edinburgh, pp 1-6

  30. Tieleman T (2008)Training restricted Boltzmann machines using approximations to the likelihood gradient. In: International conference on machine learning, Helsinki, pp 1064–1071

  31. Tieleman T, Hinton GE (2009) Using fast weights to improve persistent contrastive divergence. In: Annual international conference on machine learning, Montreal, pp 1033–1040

  32. Merino ER, Castrillejo FM, Pin JD (2018) Neighborhood-based stopping criterion for contrastive divergence. IEEE Trans Neural Netw Learn Syst 29(7):2695–2704

    MathSciNet  Google Scholar 

  33. Wang XZ, Xing HJ, Li Y et al (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654

    Google Scholar 

  34. Wang XZ, Dong CR (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567

    Google Scholar 

  35. Larochelle H, Bengio Y (2008) Classification using discriminative restricted Boltzmann machines. In: International conference on machine learning, pp 536–543

  36. Larochelle H, Mandel M, Pascanu R et al (2012) Learning algorithms for the classification restricted Boltzmann machine. J Mach Learn Res 13:643–669

    MathSciNet  MATH  Google Scholar 

  37. Elfwing S, Uchibe E, Doya K (2015) Expected energy-based restricted Boltzmann machine for classification. Neural Netw 64:29–38

    MATH  Google Scholar 

  38. Chen DD, Lv JH, Yi Z (2018) Graph regularized restricted Boltzmann machine. IEEE Trans Neural Netw Learn Syst 29(6):2651–2659

    MathSciNet  Google Scholar 

  39. Feng S, Chen CLP (2018) A fuzzy restricted Boltzmann machine: novel learning algorithms based on the crisp possibilistic mean value of fuzzy numbers. IEEE Trans Fuzzy Syst 26(1):117–130

    Google Scholar 

  40. Lee H, Grosse R, Ranganath R et al (2009) Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In: International conference on machine learning. ACM, New York, pp 609–616

  41. Ranzato M, Krizhevsky A, Hinton GE (2010) Factored 3-way restricted Boltzmann machines for modeling natural images. J Mach Learn Res 9:621–628

    Google Scholar 

  42. Srivastava N, Hinton GE, Krizhevsky A et al (2014) Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res 15(1):1929–1958

    MathSciNet  MATH  Google Scholar 

  43. Wan L, Zeiler M, Zhang S et al (2013) Regularization of neural networks using dropconnect. In: International conference on machine learning, pp 1058–1066

  44. Zhang CY, Chen CLP, Chen DW et al (2016) MapReduce based distributed learning algorithm for restricted Boltzmann machine. Neurocomputing 198:4–11

    Google Scholar 

  45. Zhang KL, Chen XW (2014) Large-scale deep belief nets with MapReduce. Access IEEE 2(2):395–403

    Google Scholar 

  46. Chen XW, Lin XT (2014) Big data deep learning: challenges and perspectives. IEEE Access 2:514–525

    Google Scholar 

  47. Zhang CX, Zhang JS, Ji NN et al (2014) Learning ensemble classifiers via restricted Boltzmann machines. Pattern Recognit Lett 36:161–170

    Google Scholar 

  48. Dean J, Ghemawat S (2008) MapReduce: simplified data processing on large clusters. Commun ACM 51(1):107–113

    Google Scholar 

  49. Hadoop. http://hadoop.apache.org/. Accessed 10 Jan 2018

  50. Frank A, Asuncion A (2013) UCI machine learning repository. http://archive.ics.uci.edu/ml. Accessed 5 Mar 2018

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China (71371063) and by the Natural Science Foundation of Hebei Province (F2017201026, F2016201161).

Author information

Authors and Affiliations

  1. Hebei Key Laboratory of Machine Learning and Computational Intelligence, College of Mathematics and Information Science, Hebei University, Baoding, 071002, Hebei, China

    Junhai Zhai & Tingting Wang

  2. College of Science, North China University of Science and Technology, Tangshan, 063210, Hebei, China

    Xu Zhou

  3. Hebei Branch of China Meteorological Administration Training Centre, China Meteorological Administration, Baoding, 071000, Hebei, China

    Sufang Zhang

Authors
  1. Junhai Zhai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xu Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sufang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Tingting Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sufang Zhang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhai, J., Zhou, X., Zhang, S. et al. Ensemble RBM-based classifier using fuzzy integral for big data classification. Int. J. Mach. Learn. & Cyber. 10, 3327–3337 (2019). https://doi.org/10.1007/s13042-019-00960-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-019-00960-3

Keywords

Search

Navigation