Skip to main content

Fuzziness-based online sequential extreme learning machine for classification problems

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

    We’re sorry, something doesn't seem to be working properly.

    Please try refreshing the page. If that doesn't work, please contact support so we can address the problem.

Abstract

The qualities of new data used in the sequential learning phase of the online sequential extreme learning machine algorithm (OS-ELM) have a significant impact on the performance of OS-ELM. This paper proposes a novel data filter mechanism for OS-ELM from the perspective of fuzziness and a fuzziness-based online sequential extreme learning machine algorithm (FOS-ELM). In FOS-ELM, when new data arrive, a fuzzy classifier first picks out the meaningful data according to the fuzziness of each sample. Specifically, the new samples with high-output fuzziness are selected and then used in sequential learning. The experimental results on eight binary classification problems and three multiclass classification problems have shown that FOS-ELM updated by the new samples with high-output fuzziness has better generalization performance than OS-ELM. Since the unimportant data are discarded before sequential learning, FOS-ELM can save more memory and have higher computational efficiency. In addition, FOS-ELM can handle data one-by-one or chunk-by-chunk with fixed or varying sizes. The relationship between the fuzziness of new samples and the model performance is also studied in this paper, which is expected to provide some useful guidelines for improving the generalization ability of online sequential learning algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Ashfaq RAR, Wang XZ (2017) Impact of fuzziness categorization on divide and conquer strategy for instance selection. J Intell Fuzzy Syst 33(2):1007–1018

    Article  Google Scholar 

  • Ashfaq RAR, He YL, Chen DG (2017a) Toward an efficient fuzziness based instance selection methodology for intrusion detection system. Int J Mach Learn Cybern 8(6):1767–1776

    Article  Google Scholar 

  • Ashfaq RAR, Wang XZ, Huang JZ, Abbas H, He YL (2017b) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497

    Article  Google Scholar 

  • Azad NL, Mozaffari A, Fathi A (2017) An optimal learning-based controller derived from Hamiltonian function combined with a cellular searching strategy for automotive coldstart emissions. Int J Mach Learn Cybern 8(3):955–979

    Article  Google Scholar 

  • Cao WP, Wang XZ, Ming Z, Gao JZ (2018) A review on neural networks with random weights. Neurocomputing 275:278–287

    Article  Google Scholar 

  • De Luca A, Termini S (1972) A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory. Inf Control 20(4):301–312

    Article  MathSciNet  MATH  Google Scholar 

  • Ding S, Zhang N, Zhang J, Xu X, Shi Z (2017) Unsupervised extreme learning machine with representational features. Int J Mach Learn Cybern 8(2):587–595

    Article  Google Scholar 

  • Feng G, Qian Z, Zhang X (2012) Evolutionary selection extreme learning machine optimization for regression. Soft Comput 16(9):1485–1491

    Article  Google Scholar 

  • Gu Y, Liu J, Chen Y, Jiang X, Yu H (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127

    Article  Google Scholar 

  • Huang GB, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern Part B (Cybern) 34(6):2284–2292

    Article  Google Scholar 

  • Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67

    Article  Google Scholar 

  • Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  • Klir GJ (1987) Where do we stand on measures of uncertainty, ambiguity, fuzziness, and the like? Fuzzy Sets Syst 24(2):141–160

    Article  MathSciNet  MATH  Google Scholar 

  • Klir GJ, Folger TA (1988) Fuzzy sets, uncertainty, and information. Prentice-Hall, Englewood Cliffs

    MATH  Google Scholar 

  • Lan Y, Soh YC, Huang GB (2009a) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395

    Article  Google Scholar 

  • Lan Y, Soh YC, Huang GB (2009b) A constructive enhancement for online sequential extreme learning machine. In: International joint conference on neural networks IJCNN 2009, IEEE, pp 1708–1713 (2009)

  • LeCun YA, Bottou L, Orr GB, Müller KR (2012) Efficient backprop. In: Neural networks: tricks of the trade, Springer, Berlin, pp 9–48

  • Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  • Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml

  • Liu M, Liu B, Zhang C, Wang W, Sun W (2017) Semi-supervised low rank kernel learning algorithm via extreme learning machine. Int J Mach Learn Cybern 8(3):1039–1052

    Article  Google Scholar 

  • Mao W, Wang J, Xue Z (2017) An ELM-based model with sparse-weighting strategy for sequential data imbalance problem. Int J Mach Learn Cybern 8(4):1333–1345

    Article  Google Scholar 

  • Meng L, Ding S, Xue Y (2017) Research on denoising sparse autoencoder. Int J Mach Learn Cybern 8(5):1719–1729

    Article  Google Scholar 

  • Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329

    Article  Google Scholar 

  • Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(4):1067–1072

    Article  Google Scholar 

  • Sánchez D, Trillas E (2012) Measures of fuzziness under different uses of fuzzy sets. In: International conference on information processing and management of uncertainty in knowledge-based systems, Springer, Berlin, pp 25–34

  • Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220

    Article  MathSciNet  Google Scholar 

  • Shao Z, Er MJ (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788

    Article  Google Scholar 

  • Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97

    Article  Google Scholar 

  • Wang GT, Li P, Cao JT (2012a) Variable activation function extreme learning machine based on residual prediction compensation. Soft Comput 16(9):1477–1484

    Article  Google Scholar 

  • Wang R, Kwong S, Wang X (2012b) A study on random weights between input and hidden layers in extreme learning machine. Soft Comput 16(9):1465–1475

    Article  Google Scholar 

  • Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015a) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654

    Article  Google Scholar 

  • Wang XZ, Ashfaq RAR, Fu AM (2015b) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196

    Article  MathSciNet  Google Scholar 

  • Wang XZ, Wang R, Xu C (2017) Discovering the relationship between generalization and uncertainty by incorporating complexity of classification. IEEE Trans Cybern 99:1–13

    Google Scholar 

  • Xie SJ, Yang J, Gong H, Yoon S, Park DS (2012) Intelligent fingerprint quality analysis using online sequential extreme learning machine. Soft Comput 16(9):1555–1568

    Article  Google Scholar 

  • Yu X, Yu H, Tian XY, Yu G, Li XM, Zhang X, Wang JY (2017) Recognition of college students from Weibo with deep neural networks. Int J Mach Learn Cybern 8(5):1447–1455

    Article  Google Scholar 

  • Zadeh LA (1968) Probability measures of fuzzy events. J Math Anal Appl 23(2):421–427

    Article  MathSciNet  MATH  Google Scholar 

  • Zhai JH, Xu HY, Wang XZ (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502

    Article  Google Scholar 

  • Zhai J, Zhang S, Wang C (2017) The classification of imbalanced large data sets based on mapreduce and ensemble of elm classifiers. Int J Mach Learn Cybern 8(3):1009–1017

    Article  Google Scholar 

  • Zhang HG, Zhang S, Yin YX (2017) Online sequential ELM algorithm with forgetting factor for real applications. Neurocomputing 261:144–152

    Article  Google Scholar 

  • Zhao JW, Wang Z, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87:79–89

    Article  Google Scholar 

  • Zhu HY, Wang XZ (2017) A cost-sensitive semi-supervised learning model based on uncertainty. Neurocomputing 251:106–114

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the National Natural Science Foundation of China under Grant No. 61672358, and in part by Guangdong province 2014GKXM054.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Zhong Ming.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Informed consent

Informed consent was obtained from all individual participants included in the study.

Additional information

Communicated by X. Wang, A. K. Sangaiah and M. Pelillo.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cao, W., Gao, J., Ming, Z. et al. Fuzziness-based online sequential extreme learning machine for classification problems. Soft Comput 22, 3487–3494 (2018). https://doi.org/10.1007/s00500-018-3021-4

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-018-3021-4

Keywords