Abstract
The qualities of new data used in the sequential learning phase of the online sequential extreme learning machine algorithm (OS-ELM) have a significant impact on the performance of OS-ELM. This paper proposes a novel data filter mechanism for OS-ELM from the perspective of fuzziness and a fuzziness-based online sequential extreme learning machine algorithm (FOS-ELM). In FOS-ELM, when new data arrive, a fuzzy classifier first picks out the meaningful data according to the fuzziness of each sample. Specifically, the new samples with high-output fuzziness are selected and then used in sequential learning. The experimental results on eight binary classification problems and three multiclass classification problems have shown that FOS-ELM updated by the new samples with high-output fuzziness has better generalization performance than OS-ELM. Since the unimportant data are discarded before sequential learning, FOS-ELM can save more memory and have higher computational efficiency. In addition, FOS-ELM can handle data one-by-one or chunk-by-chunk with fixed or varying sizes. The relationship between the fuzziness of new samples and the model performance is also studied in this paper, which is expected to provide some useful guidelines for improving the generalization ability of online sequential learning algorithms.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
Ashfaq RAR, Wang XZ (2017) Impact of fuzziness categorization on divide and conquer strategy for instance selection. J Intell Fuzzy Syst 33(2):1007–1018
Ashfaq RAR, He YL, Chen DG (2017a) Toward an efficient fuzziness based instance selection methodology for intrusion detection system. Int J Mach Learn Cybern 8(6):1767–1776
Ashfaq RAR, Wang XZ, Huang JZ, Abbas H, He YL (2017b) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497
Azad NL, Mozaffari A, Fathi A (2017) An optimal learning-based controller derived from Hamiltonian function combined with a cellular searching strategy for automotive coldstart emissions. Int J Mach Learn Cybern 8(3):955–979
Cao WP, Wang XZ, Ming Z, Gao JZ (2018) A review on neural networks with random weights. Neurocomputing 275:278–287
De Luca A, Termini S (1972) A definition of a nonprobabilistic entropy in the setting of fuzzy sets theory. Inf Control 20(4):301–312
Ding S, Zhang N, Zhang J, Xu X, Shi Z (2017) Unsupervised extreme learning machine with representational features. Int J Mach Learn Cybern 8(2):587–595
Feng G, Qian Z, Zhang X (2012) Evolutionary selection extreme learning machine optimization for regression. Soft Comput 16(9):1485–1491
Gu Y, Liu J, Chen Y, Jiang X, Yu H (2014) TOSELM: timeliness online sequential extreme learning machine. Neurocomputing 128:119–127
Huang GB, Saratchandran P, Sundararajan N (2004) An efficient sequential learning algorithm for growing and pruning RBF (GAP-RBF) networks. IEEE Trans Syst Man Cybern Part B (Cybern) 34(6):2284–2292
Huang GB, Saratchandran P, Sundararajan N (2005) A generalized growing and pruning RBF (GGAP-RBF) neural network for function approximation. IEEE Trans Neural Netw 16(1):57–67
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892
Klir GJ (1987) Where do we stand on measures of uncertainty, ambiguity, fuzziness, and the like? Fuzzy Sets Syst 24(2):141–160
Klir GJ, Folger TA (1988) Fuzzy sets, uncertainty, and information. Prentice-Hall, Englewood Cliffs
Lan Y, Soh YC, Huang GB (2009a) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13):3391–3395
Lan Y, Soh YC, Huang GB (2009b) A constructive enhancement for online sequential extreme learning machine. In: International joint conference on neural networks IJCNN 2009, IEEE, pp 1708–1713 (2009)
LeCun YA, Bottou L, Orr GB, Müller KR (2012) Efficient backprop. In: Neural networks: tricks of the trade, Springer, Berlin, pp 9–48
Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423
Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml
Liu M, Liu B, Zhang C, Wang W, Sun W (2017) Semi-supervised low rank kernel learning algorithm via extreme learning machine. Int J Mach Learn Cybern 8(3):1039–1052
Mao W, Wang J, Xue Z (2017) An ELM-based model with sparse-weighting strategy for sequential data imbalance problem. Int J Mach Learn Cybern 8(4):1333–1345
Meng L, Ding S, Xue Y (2017) Research on denoising sparse autoencoder. Int J Mach Learn Cybern 8(5):1719–1729
Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329
Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern Part B (Cybern) 39(4):1067–1072
Sánchez D, Trillas E (2012) Measures of fuzziness under different uses of fuzzy sets. In: International conference on information processing and management of uncertainty in knowledge-based systems, Springer, Berlin, pp 25–34
Scardapane S, Comminiello D, Scarpiniti M, Uncini A (2015) Online sequential extreme learning machine with kernels. IEEE Trans Neural Netw Learn Syst 26(9):2214–2220
Shao Z, Er MJ (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788
Wang XY, Han M (2014) Online sequential extreme learning machine with kernels for nonstationary time series prediction. Neurocomputing 145:90–97
Wang GT, Li P, Cao JT (2012a) Variable activation function extreme learning machine based on residual prediction compensation. Soft Comput 16(9):1477–1484
Wang R, Kwong S, Wang X (2012b) A study on random weights between input and hidden layers in extreme learning machine. Soft Comput 16(9):1465–1475
Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015a) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654
Wang XZ, Ashfaq RAR, Fu AM (2015b) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196
Wang XZ, Wang R, Xu C (2017) Discovering the relationship between generalization and uncertainty by incorporating complexity of classification. IEEE Trans Cybern 99:1–13
Xie SJ, Yang J, Gong H, Yoon S, Park DS (2012) Intelligent fingerprint quality analysis using online sequential extreme learning machine. Soft Comput 16(9):1555–1568
Yu X, Yu H, Tian XY, Yu G, Li XM, Zhang X, Wang JY (2017) Recognition of college students from Weibo with deep neural networks. Int J Mach Learn Cybern 8(5):1447–1455
Zadeh LA (1968) Probability measures of fuzzy events. J Math Anal Appl 23(2):421–427
Zhai JH, Xu HY, Wang XZ (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502
Zhai J, Zhang S, Wang C (2017) The classification of imbalanced large data sets based on mapreduce and ensemble of elm classifiers. Int J Mach Learn Cybern 8(3):1009–1017
Zhang HG, Zhang S, Yin YX (2017) Online sequential ELM algorithm with forgetting factor for real applications. Neurocomputing 261:144–152
Zhao JW, Wang Z, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87:79–89
Zhu HY, Wang XZ (2017) A cost-sensitive semi-supervised learning model based on uncertainty. Neurocomputing 251:106–114
Acknowledgements
This research is supported by the National Natural Science Foundation of China under Grant No. 61672358, and in part by Guangdong province 2014GKXM054.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Informed consent
Informed consent was obtained from all individual participants included in the study.
Additional information
Communicated by X. Wang, A. K. Sangaiah and M. Pelillo.
Rights and permissions
About this article
Cite this article
Cao, W., Gao, J., Ming, Z. et al. Fuzziness-based online sequential extreme learning machine for classification problems. Soft Comput 22, 3487–3494 (2018). https://doi.org/10.1007/s00500-018-3021-4
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00500-018-3021-4