Skip to main content
Springer Nature Link
Log in

A Novel Online Sequential Learning Algorithm for ELM Based on Optimal Control

  • Conference paper
  • First Online:
Knowledge Science, Engineering and Management (KSEM 2024)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14885))

  • 411 Accesses

Abstract

Aiming to address the deficiency in Extreme Learning Machine (ELM), particularly its ineffectiveness in handling data streaming scenarios and the necessity for retraining upon receiving new data after the model has been fitted, this paper introduces a novel algorithm designed to update ELM parameters online. The algorithm incorporates the concept of optimal control into the training of machine learning models, formulating the ELM output weights calculation problem as a series of state feedback control problems within a control system framework. This is addressed through the application of the Online Linear Quadratic Regulator (OLQR). The proposed algorithm demonstrates rapid and robust convergence, leveraging the advantages of optimal control technology. Moreover, the algorithm incorporates a regularization term into the quadratic objective function. This addition not only ensures high performance but also effectively mitigates overfitting. Extensive experimentation on UCI benchmark datasets substantiates that the proposed algorithm achieves faster convergence and superior generalization performance compared to the mainstream recursive least-squares-based online learning method. The code is available at https://www.gitlink.org.cn/BIT2024/OLQR-ELM/tree/master.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: a new learning scheme of feedforward neural networks. In: 2004 Ieee International Joint Conference On Neural Networks (IEEE Cat. No. 04CH37541), vol. 2, pp. 985–990. IEEE (2004)

    Google Scholar 

  2. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)

    Article  Google Scholar 

  3. Gantmacher, F.R., Brenner, J.L.: Applications of the Theory of Matrices. Courier Corporation (2005)

    Google Scholar 

  4. Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybernetics) 42(2), 513–529 (2011)

    Google Scholar 

  5. Xu, H.: Performance enhancement of kernel extreme learning machine using whale optimization algorithm in fruit image classification. In: 2023 International Conference on Advanced Mechatronic Systems (ICAMechS), pp. 1–6. IEEE (2023)

    Google Scholar 

  6. Daneshfar, F., Kabudian, S.J.: Speech emotion recognition using multi-layer sparse auto-encoder extreme learning machine and spectral/spectro-temporal features with new weighting method for data imbalance. In: 2021 11th International Conference on Computer Engineering and Knowledge (ICCKE), pp. 419–423. IEEE (2021)

    Google Scholar 

  7. Goni, M.O.F., Mondal, M.N.I., Islam, S.R., Nahiduzzaman, M., Islam, M.R., Anower, M.S., Kwak, K.S.: Diagnosis of malaria using double hidden layer extreme learning machine algorithm with cnn feature extraction and parasite inflator. IEEE Access 11, 4117–4130 (2023)

    Article  Google Scholar 

  8. Deng, W., Zheng, Q., Chen, L.: Regularized extreme learning machine. In: 2009 IEEE Symposium on Computational Intelligence and Data Mining, pp. 389–395. IEEE (2009)

    Google Scholar 

  9. Tang, Y., Li, C.: An online network intrusion detection model based on improved regularized extreme learning machine. IEEE Access 9, 94826–94844 (2021)

    Article  Google Scholar 

  10. Sun, A., Wei, F., Wang, G., Li, Y.: Chinese sentiment analysis using regularized extreme learning machine and stochastic optimization. In: 2022 4th International Conference on Natural Language Processing (ICNLP), pp. 525–529. IEEE (2022)

    Google Scholar 

  11. Cai, W., Yang, J., Yu, Y., Song, Y., Zhou, T., Qin, J.: Pso-elm: a hybrid learning model for short-term traffic flow forecasting. IEEE Access 8, 6505–6514 (2020)

    Article  Google Scholar 

  12. Chen, X., Xu, L., Wang, Y., Zhai, X., Guo, X.: Application of improved pso-elm in auto insurance customer risk level prediction. In: 2020 IEEE International Conference on Mechatronics and Automation (ICMA), pp. 979–983. IEEE (2020)

    Google Scholar 

  13. Zhang, S., Zhou, W.: Prediction model of heart failure disease based on ga-elm. In: 2021 40th Chinese Control Conference (CCC), pp. 7944–7948. IEEE (2021)

    Google Scholar 

  14. Zhu, J., Tan, T., Wu, L., Yuan, H.: Rul prediction of lithium-ion battery based on improved dgwo-elm method in a random discharge rates environment. Ieee Access 7, 125176–125187 (2019)

    Article  Google Scholar 

  15. Wang, X.B., Zhang, X., Li, Z., Wu, J.: Ensemble extreme learning machines for compound-fault diagnosis of rotating machinery. Knowl.-Based Syst. 188, 105012 (2020)

    Article  Google Scholar 

  16. Abd Shehab, M., Kahraman, N.: A weighted voting ensemble of efficient regularized extreme learning machine. Comput. Electr. Eng. 85, 106639 (2020)

    Article  Google Scholar 

  17. Li, Y., Zhang, J., Zhang, S., Xiao, W., Zhang, Z.: Multi-objective optimization-based adaptive class-specific cost extreme learning machine for imbalanced classification. Neurocomputing 496, 107–120 (2022)

    Article  Google Scholar 

  18. Wang, K., Bian, X., Zheng, M., Liu, P., Lin, L., Tan, X.: Rapid determination of hemoglobin concentration by a novel ensemble extreme learning machine method combined with near-infrared spectroscopy. Spectrochim. Acta Part A Mol. Biomol. Spectrosc. 263, 120138 (2021)

    Article  Google Scholar 

  19. Liang, N.Y., Huang, G.B., Saratchandran, P., Sundararajan, N.: A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans. Neural Netw. 17(6), 1411–1423 (2006)

    Article  Google Scholar 

  20. Liu, Y., Cao, W., Liu, Y., Li, D., Wang, Q.: Ensemble online sequential extreme learning machine for air quality prediction. In: 2021 IEEE 7th International Conference on Control Science and Systems Engineering (ICCSSE), pp. 233–237. IEEE (2021)

    Google Scholar 

  21. Zhu, Q., Bai, R., Li, M., Chen, S., Wen, P.: Bayes-os-elm: An novel ensemble method for classification application. In: 2019 International Conference on Sensing, Diagnostics, Prognostics, and Control (SDPC), pp. 160–166. IEEE (2019)

    Google Scholar 

  22. Yu, H., Xie, H., Yang, X., Zou, H., Gao, S.: Online sequential extreme learning machine with the increased classes. Comput. Electr. Eng. 90, 107008 (2021)

    Article  Google Scholar 

  23. Ning, H., Zhang, J., Jing, X., Tian, T.: Robust online learning method based on dynamical linear quadratic regulator. IEEE Access 7, 117780–117795 (2019)

    Article  Google Scholar 

  24. Ning, H., Zhang, J., Feng, T.T., Chu, E.K.w., Tian, T.: Control-based algorithms for high dimensional online learning. J. Franklin Institute 357(3), 1909–1942 (2020)

    Google Scholar 

  25. Lewis, F.L., Vrabie, D., Syrmos, V.L.: Optimal control. John Wiley & Sons (2012)

    Google Scholar 

  26. Asuncion, A., Newman, D.: Uci machine learning repository (2007)

    Google Scholar 

  27. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported by the Open Research Fund of Anhui Province Key Laboratory of Machine Vision Inspection (KLMVI-2023-HIT-20).

Author information

Authors and Affiliations

  1. School of Automation, Beijing Institute of Technology, Beijing, 100081, China

    Huihuang Lu, Weidong Zou & Liping Yan

Authors
  1. Huihuang Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Weidong Zou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liping Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weidong Zou .

Editor information

Editors and Affiliations

  1. Chinese Academy of Sciences, Beijing, China

    Cungeng Cao

  2. Zhejiang University, Zhejiang, China

    Huajun Chen

  3. Emory University, Atlanta, GA, USA

    Liang Zhao

  4. Birmingham City University, Birmingham, UK

    Junaid Arshad

  5. Monash University, Banten, Indonesia

    Taufiq Asyhari

  6. Birmingham City University, Birmingham, UK

    Yonghao Wang

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lu, H., Zou, W., Yan, L. (2024). A Novel Online Sequential Learning Algorithm for ELM Based on Optimal Control. In: Cao, C., Chen, H., Zhao, L., Arshad, J., Asyhari, T., Wang, Y. (eds) Knowledge Science, Engineering and Management. KSEM 2024. Lecture Notes in Computer Science(), vol 14885. Springer, Singapore. https://doi.org/10.1007/978-981-97-5495-3_8

Download citation

  • DOI: https://doi.org/10.1007/978-981-97-5495-3_8

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-97-5494-6

  • Online ISBN: 978-981-97-5495-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics