Skip to main content
SpringerLink
Log in

Parallelized extreme learning machine for online data classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The challenges raised by the massive data are being managed by the community through the advancements of infrastructure and algorithms, and now the processing of fast data is becoming a new hurdle to the researchers. Extreme Learning Machine (ELM) is a single-layer learning model with reliable performances and it is computationally simpler than the new generation deep architectures. ELM process the data in batches and the model has to be rerun while updates happening in the datasets. In the theoretical background of ELM, the past knowledge cannot be reused for improving the performance in online learning where the data set will be updated with mini-batches. In this paper, we have introduced a knowledge base to deal with the remembrance of knowledge in ELM. The architecture of the proposed model is designed to process mini-batches of any size to speed up the processing of the data on its arrival. A group of data sets with different properties such as sparse and feature dimensions is used in the experiments to evaluate our method. The performance of the algorithm is compared with a set of benchmarked classifiers and stream classifiers in the scikit-learn public platform. It is observed that our method could perform better in most of the experiments. It clear in the results that the Parallel ELM model outperformed the other methods in the training time across all the datasets. The consistent performance of our method shows the significance of parallel algorithms of ELM that can remember past knowledge.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 IEEE international joint conference on neural networks (IEEE Cat. No. 04CH37541), vol 2. IEEE, pp 985–990

  2. Bifet A, Holmes G, Pfahringer B, Kranen P, Kremer H, Jansen T, Seidl T (2010) Moa: Massive online analysis, a framework for stream classification and clustering. In: Proceedings of the first workshop on applications of pattern analysis. PMLR, pp 44–50

  3. Dua D, Graff C (2017) UCI machine learning repository

  4. Bottou L (2010) Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMPSTAT’2010. Physica-Verlag HD, pp 177–186

  5. Chang YW, Hsieh CJ, Chang KW, Ringgaard M, Lin CJ (2010) Training and testing low-degree polynomial data mappings via linear SVM. J Mach Learn Res, 11(4)

  6. Yang X (2020) Introduction to stochastic calculus and its applications. Available at SSRN 3607647

  7. Ho TK (1995) Random decision forests. In: Proceedings of 3rd international conference on document analysis and recognition, vol 1. IEEE, pp 278–282

  8. Hastie T, Tibshirani R, Friedman J (2009) The elements of statistical learning: data mining, inference, and prediction. Springer Science & Business Media, Berlin

    Book  Google Scholar 

  9. Kégl B (2013) The return of AdaBoost. MH: multi-class Hamming trees. arXiv:1312.6086

  10. John GH, Langley P (2013) Estimating continuous distributions in Bayesian classifiers. arXiv:1302.4964

  11. Tharwat A (2016) Linear vs. quadratic discriminant analysis classifier: a tutorial. Int J Appl Pattern Recognit 3(2):145–180

    Article  Google Scholar 

  12. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, et al. (2011) Scikit-learn: Machine learning in Python. J Machine Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  13. Kolter JZ, Maloof MA (2007) Dynamic weighted majority: an ensemble method for drifting concepts. J Machine Learn Res 8:2755–2790

    MATH  Google Scholar 

  14. Wang R, Chow CY, Lyu Y, Lee VC, Kwong S, Li Y, Zeng J (2017) Taxirec: recommending road clusters to taxi drivers using ranking-based extreme learning machines. IEEE Trans Knowl Data Eng 30(3):585–598

    Article  Google Scholar 

  15. Wang H, Fan W, Yu PS, Han J (2003) Mining concept-drifting data streams using ensemble classifiers. In: Proceedings of the ninth ACM SIGKDD international conference on knowledge discovery and data mining, pp 226–235

  16. Wang B, Pineau J (2016) Online bagging and boosting for imbalanced data streams. IEEE Trans Knowl Data Eng 28(12):3353–3366

    Article  Google Scholar 

  17. Oza NC, Russell SJ (2001) Online bagging and boosting. In: International workshop on artificial intelligence and statistics. PMLR, pp 229–236

  18. Montiel J, Read J, Bifet A, Abdessalem T (2018) Scikit-multiflow: A multi-output streaming framework. J Machine Learn Res 19(1):2915–2914

    Google Scholar 

  19. Vanschoren J, Van Rijn JN, Bischl B, Torgo L (2014) OpenML: networked science in machine learning. ACM SIGKDD Explorations Newsletter 15(2):49–60. https://doi.org/10.1145/2641190.2641198

    Article  Google Scholar 

  20. Duan J, Ou Y, Hu J, Wang Z, Jin S, Xu C (2017) Fast and stable learning of dynamical systems based on extreme learning machine. IEEE Trans Syst Man Cybern Syst 49(6):1175–1185

    Article  Google Scholar 

  21. Gomes HM, Bifet A, Read J, Barddal JP, Enembreck F, Pfharinger B, Holmes G, Abdessalem T (2017) Adaptive random forests for evolving data stream classification. Mach Learn 106(9):1469–1495

    Article  MathSciNet  Google Scholar 

  22. Kumar S, Banerjee B, Chaudhuri S (2021) Improved landcover classification using online spectral data hallucination. Neurocomputing 439:316–326

    Article  Google Scholar 

  23. Dadkhah S, Shoeleh F, Yadollahi MM, Zhang X, Ghorbani AA (2021) A real-time hostile activities analyses and detection system. Applied Soft Computing 104:107175

    Article  Google Scholar 

  24. Seraphim BI, Poovammal E (2021) Adversarial attack by inducing drift in streaming data. Wirel Pers Commun, 1–25

  25. Li K, Luo G, Ye Y, Li W, Ji S, Cai Z (2020) Adversarial Privacy Preserving Graph Embedding against Inference Attack. IEEE Internet of Things Journal

  26. Dong Y, Yang C, Zhang Y (2021) Deep metric learning with online hard mining for hyperspectral classification. Remote Sens 13(7):1368

    Article  Google Scholar 

  27. Jo K, Kim J, Kim D, Jang C, Sunwoo M (2014) Development of autonomous car—Part I: Distributed system architecture and development process. IEEE Trans Ind Electron 61(12):7131–7140

    Article  Google Scholar 

  28. Mirza B, Lin Z, Toh KA (2013) Weighted online sequential extreme learning machine for class imbalance learning. Neural Process Lett 38(3):465–486

    Article  Google Scholar 

  29. Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  30. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16-18):3460–3468

    Article  Google Scholar 

  31. Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  32. Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411– 1423

    Article  Google Scholar 

  33. Yu H, Xie H, Yang X, Zou H, Gao S (2021) Online sequential extreme learning machine with the increased classes. Comput Electric Eng 90:107008

    Article  Google Scholar 

  34. Wu C, Khishe M, Mohammadi M, Karim SHT, Rashid TA (2021) Evolving deep convolutional neutral network by hybrid sine–cosine and extreme learning machine for real-time COVID19 diagnosis from X-ray images. Soft Comput, 1–20

  35. Rathore S, Park JH (2018) Semi-supervised learning based distributed attack detection framework for IoT. Appl Soft Comput 72:79–89

    Article  Google Scholar 

  36. Zhang X, He T, Lu L, Yue S, Cheng D, Xu X (2017) Video analysis of traffic accidents based on projection extreme learning machine. In: 2017 international symposium on intelligent signal processing and communication systems (ISPACS). IEEE, pp 149–154

  37. Ghomeshi H, Gaber MM, Kovalchuk Y (2020) A non-canonical hybrid metaheuristic approach to adaptive data stream classification. Futur Gener Comput Syst 102:127–139

    Article  Google Scholar 

  38. Ghomeshi H, Gaber MM, Kovalchuk Y (2019) EACD: Evolutionary Adaptation to concept drifts in data streams. Data Min Knowl Disc 33(3):663–694

    Article  Google Scholar 

  39. Liu W, Zhang H, Ding Z, Liu Q, Zhu C (2021) A comprehensive active learning method for multiclass imbalanced data streams with concept drift. Knowledge-Based Systems 215:106778

    Article  Google Scholar 

  40. Li Z, Huang W, Xiong Y, Ren S, Zhu T (2020) Incremental learning imbalanced data streams with concept drift: The dynamic updated ensemble algorithm. Knowledge-Based Systems 195:105694

    Article  Google Scholar 

  41. Baidari I, Honnikoll N (2020) Accuracy weighted diversity-based online boosting. Expert Systems with Applications 160:113723

    Article  Google Scholar 

  42. Sarnovsky M, Kolarik M (2021) Classification of the drifting data streams using heterogeneous diversified dynamic class-weighted ensemble. PeerJ Computer Science 7:e459

    Article  Google Scholar 

  43. Museba T, Nelwamondo F, Ouahada K, Akinola A (2021) Recurrent adaptive classifier ensemble for handling recurring concept drifts. Applied Computational Intelligence and Soft Computing, 2021

  44. Aydogdu O, Ekinci M (2020) A new approach for data stream classification: unsupervised feature representational online sequential extreme learning machine. Multimed Tools Appl 79(37):27205–27227

    Article  Google Scholar 

  45. Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72(13-15):3391–3395

    Article  Google Scholar 

  46. Xu S, Wang J (2016) A fast incremental extreme learning machine algorithm for data streams classification. Expert Syst Appl 65:332–344

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Kerala, Thiruvananthapuram, Kerala, India

    Vidhya M & Aji S

Authors
  1. Vidhya M
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aji S
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Vidhya M.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

M, V., S, A. Parallelized extreme learning machine for online data classification. Appl Intell 52, 14164–14177 (2022). https://doi.org/10.1007/s10489-022-03308-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03308-7

Keywords

Search

Navigation