Skip to main content
SpringerLink
Log in

A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

The characteristics of the data stream have brought enormous challenges to classification algorithms. Concept drift is the most concerning characteristics, and developed classification algorithms must tackle the concept drift problem. Therefore, Extreme Learning Machines (ELM) based algorithms have been developed to respond to the characteristics of the data stream. However, due to randomly assigned input layer weights, ELM based algorithms have encountered problems such as producing inconsistent outputs, generating ill-conditioned matrix, and mapping the inputs to the worst representative space. To overcome these problems, this paper aims to build a stable and well-constructed classifier that responds to the requirements of the data stream by considering all characteristics. A novel data stream classification approach based online sequential ELM (OS-ELM) with unsupervised feature representation learning (UFROS-ELM) and ensemble UFROS-ELM approach based on majority learning is presented in this paper. UFROS-ELM is a modification of the OS-ELM with ELM-AE and concept drift mechanism. ELM-AE is utilized for computing the best discriminative input weights of the classifier. The classifier is then initialized by using the determined weights, first chunk, and OS-ELM algorithm. When a new data chunk arrives, the classifier firstly searches any concept drift occurrence. If it is detected, ELM-AE is utilized to reconstruct the classifier to adapt to the changes. Otherwise, the classifier is sequentially updated updates by processing the current chunk. The results are achieved on the well-known real and artificial data sets and compared with state-of-the-art data stream classification algorithms. The experimental studies demonstrate the achievements of the proposed approaches.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Aggarwal CC (2014) Data classification: algorithms and applications. CRC Press, Boca Raton

    Book  Google Scholar 

  2. Amini A, Saboohi H, Ying wah t, Herawan T (2014) A fast density-based clustering algorithm for real-time internet of things stream. The Scientific World Journal. https://doi.org/10.1155/2014/926020

  3. Arabmakki E, Kantardzic M (2017) SOM-based partial labeling of imbalanced data stream. Neurocomputing 262:120–133. https://doi.org/10.1016/j.neucom.2016.11.088

    Article  Google Scholar 

  4. Armbrust M, Fox A, Griffith R (2010) A view of cloud computing. Commun ACM 53(4):50–58. https://doi.org/10.1145/1721654.1721672

    Article  Google Scholar 

  5. Atzori L, Iera A, Morabito G (2010) The internet of things: a survey. Comput Netw 54(15):2787–2805. https://doi.org/10.1016/j.comnet.2010.05.010

    Article  MATH  Google Scholar 

  6. Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. NIPS’06 Proceedings of the 19th international conference on neural information processing systems 153–160

  7. Bifet A, Holmes G, Kirkby R, Pfahringer B (2010) Moa: Massive online analysis. J Mach Learn Res 11:1601–1604

    Google Scholar 

  8. Castaño A, Fernández-Navarro F, Hervás-Martínez C (2013) PCA-ELM: A robust and pruned extreme learning machine approach based on principal component analysis. Neur Process Lett 37(3):37–392

    Google Scholar 

  9. Deng W-Y, Ong Y-S, Tan PS, et al. (2016) Online sequential reduced kernel extreme learning machine. Neurocomputing 174:72–84. https://doi.org/10.1016/j.neucom.2015.06.087

    Article  Google Scholar 

  10. Ding S, Mirza B, Lin Z, et al. (2018) Kernel based online learning for imbalance multiclass classification. Neurocomputing 277:139–148

    Article  Google Scholar 

  11. Ding S, Zhang N, Zhang J, Xu X, Shi Z (2017) Unsupervised extreme learning machine with representational features. Int J Mach Learn Cyber 8(2):587–595

    Article  Google Scholar 

  12. Dua D, Karra TE (2017) UCI machine learning repository

  13. Fu Z, Sun X, Liu Q, Zhou L, Shu J (2015) Achieving efficient cloud search services: multi-keyword ranked search over encrypted cloud data supporting parallel computing. IEICE Trans Commun E98(B(1)):190–200. https://doi.org/10.1587/transcom.E98.B.190

    Article  Google Scholar 

  14. Han M, Liu XX (2014) An extreme learning machine algorithm based on mutual information variable selection. Control Decision 29(9):1576–1580

    MATH  Google Scholar 

  15. Han F, Yao HF, Ling QH (2013) An improved evolutionary extreme learning machine based on particle swarm optimization. Neurocomputing 116:87–93

    Article  Google Scholar 

  16. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cyber Part B (Cybernetics) 42(2):513–529

    Article  Google Scholar 

  17. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1-3):489–501. https://doi.org/10.1016/j.neucom.2005.12.126

    Article  Google Scholar 

  18. Kasun LLC, Zhou H, Huang GB (2013) Representational learning with ELMs for big data. IEEE Intell Syst 28(6):31–34

    Google Scholar 

  19. Krawczyk B, Cano A (2018) Online ensemble learning with abstaining classifiers for drifting and noisy data streams. Appl Soft Comput 68:677–692. https://doi.org/10.1016/j.asoc.2017.12.008

    Article  Google Scholar 

  20. Lall A, Sekar V, Ogihara M, Xu J, Zhang H (2006) Data streaming algorithms for estimating entropy of network traffic. ACM Sigmet Perform Eval Rev 34(1):145–156. https://doi.org/10.1145/1140103.1140295

    Article  Google Scholar 

  21. Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13-15):3391–3395

    Article  Google Scholar 

  22. Laohakiat S, Phimoltares S, Lursinsap C (2017) A clustering algorithm for stream data with LDA-based unsupervised localized dimension reduction. Inf Sci 381:104–123

    Article  Google Scholar 

  23. Li L, Sun R, Cai S, Zhao K, Zhang Q (2019) A review of improved extreme learning machine methods for data stream classification. Multimed Tools Appl 1–26 https://doi.org/10.1007/s11042-019-7543-2

  24. Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neur Netw 17(6):1411–1423. https://doi.org/10.1109/TNN.2006.880583

    Article  Google Scholar 

  25. Liu Z, Loo CK, Seera M (2019) Meta-cognitive recurrent recursive kernel OS-ELM for concept drift handling. Appl Soft Comput 75:494–507

    Article  Google Scholar 

  26. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2019) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neur Netw 21(1):158–162

    Article  Google Scholar 

  27. Mirza B, Lin Z (2006) Meta-cognitive online sequential extreme learning machine for imbalanced and concept-drifting data classification. Neur Netw 80:79–94

    Article  Google Scholar 

  28. Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–29

    Article  Google Scholar 

  29. Mirza B, Lin Z, Toh KA (2013) Weighted online sequential extreme learning machine for class imbalance learning. Neural processing letters 38(3):465–486

    Article  Google Scholar 

  30. Pacheco AG, Krohling RA, da Silva CA (2018) Restricted Boltzmann machine to determine the input weights for extreme learning machines. Expert Syst Appl 96:77–85

    Article  Google Scholar 

  31. Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York

    MATH  Google Scholar 

  32. Rutkowski L, Pietruczuk L, Duda P, Jaworski M (2013) Decision trees for mining data streams based on the McDiarmid’s bound. IEEE Trans Knowl Data Eng 25(6):1272–1279

    Article  Google Scholar 

  33. Sethi TS, Kantardzic M (2017) On the reliable detection of concept drift from streaming unlabeled data. Expert Syst Appl 82:77–99

    Article  Google Scholar 

  34. Shao Z, Er MJ (2016) An online sequential learning algorithm for regularized extreme learning machine. Neurocomputing 173:778–788. https://doi.org/10.1016/j.neucom.2015.08.029

    Article  Google Scholar 

  35. Singh R, Kumar H, Singla RK (2015) An intrusion detection system using network traffic profiling and online sequential extreme learning machine. Expert Syst Appl 42(22):8609–8624

    Article  Google Scholar 

  36. Tso F, Cui L, Zhang L (2013) Dragonnet: a robust mobile internet service system for long-distance trains. IEEE Trans Mob Comput 12(11):2206–2218. https://doi.org/10.1109/TMC.2012.191

    Article  Google Scholar 

  37. Venkatesan R, Er MJ, Dave M, Pratama M, Wu S (2017) A novel online multi-label classifier for high-speed streaming data applications. Evolving Syst 8(4):303–315

    Article  Google Scholar 

  38. Venkatesan R, Er MJ, Wu S, Pratama M (2016) A novel online real-time classifier for multi-label data streams. In: Proceedings International Joint Conference on Neural Network (IJCNN), Vancouver, BC Canada, pp 1833–1840

  39. Wang Y, Cao F, Yuan Y (2011) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490. https://doi.org/10.1016/j.neucom.2010.11.030

    Article  Google Scholar 

  40. Wang W, Liu X (2017) The selection of input weights of extreme learning machine: a sample structure preserving point of view. Neurocomputing 261:28–36

    Article  Google Scholar 

  41. Wang D, Wang P, Ji Y (2015) An oscillation bound of the generalization performance of extreme learning machine and corresponding analysis. Neurocomputing 151:883–890

    Article  Google Scholar 

  42. Webb GI, Hyde R, Cao H, Nguyen HL, Petitjean F (2016) Characterizing concept drift. Data Min Knowl Disc 30(4):964–994. https://doi.org/10.1007/s10618-015-0448-4

    Article  MathSciNet  MATH  Google Scholar 

  43. Xiao D, Li B, Zhang S (2018) An online sequential multiple hidden layers extreme learning machine method with forgetting mechanism. Chemometr Intell Lab Syst 176:126–133

    Article  Google Scholar 

  44. Xu S, Wang J (2016) A fast incremental extreme learning machine algorithm for data streams classification. Expert Syst Appl 65:332–344. https://doi.org/10.1016/j.eswa.2016.08.052

    Article  Google Scholar 

  45. Xu S, Wang J (2017) Dynamic extreme learning machine for data stream classification. Neurocomputing 238:433–449

    Article  Google Scholar 

  46. Yang R, Xu S, Feng L (2018) An ensemble extreme learning machine for data stream classification. Algorithms 11(7):107

    Article  MathSciNet  Google Scholar 

  47. Yu H, Webb GI (2019) Adaptive online extreme learning machine by regulating forgetting factor by concept drift map. Neurocomputing 343:141–153

    Article  Google Scholar 

  48. Zeng XQ, Li GZ (2014) Incremental partial least squares analysis of big streaming data. Pattern Recogn 47(11):3726–3735. https://doi.org/10.1016/j.patcog.2014.05.022

    Article  Google Scholar 

  49. Zeng Y, Qian L, Ren J (2018) Evolutionary hierarchical sparse extreme learning autoencoder network for object recognition. Symmetry 10(10):474

    Article  Google Scholar 

  50. Zhang Y, Liu W, Ren X, et al. (2017) Dual weighted extreme learning machine for imbalanced data stream classification. J Intell Fuzzy Syst 33 (2):1143–1154

    Article  Google Scholar 

  51. Zhang P, Zhu X, Shi Y, Guo L, Wu X (2011) Robust ensemble learning for mining noisy data streams. Decis Support Syst 50(2):469–479

    Article  Google Scholar 

  52. Zhao G, Shen Z, Man Z (2011) Robust input weight selection for well-conditioned extreme learning machine. Int J Inf Technol 17(1):1–13

    Google Scholar 

  53. Zhao J, Wang Z, Park DS (2012) Online sequential extreme learning machine with forgetting mechanism. Neurocomputing 87:79–89

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Karadeniz Technical University, 61080, Trabzon, Turkey

    Ozge Aydogdu & Murat Ekinci

Authors
  1. Ozge Aydogdu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Murat Ekinci
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ozge Aydogdu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Aydogdu, O., Ekinci, M. A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine. Multimed Tools Appl 79, 27205–27227 (2020). https://doi.org/10.1007/s11042-020-09300-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09300-y

Keywords

Search

Navigation