Skip to main content
SpringerLink
Log in

Incremental Extreme Learning Machine via Fast Random Search Method

  • Conference paper
  • First Online:
Neural Information Processing (ICONIP 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10634))

Included in the following conference series:

Abstract

Since extreme learning machine (ELM) was proposed, it has been found that some hidden nodes in ELM may play a very minor role in the network output. To avoid this problem, enhanced random search based incremental extreme learning machine (EI-ELM) is proposed. However, we find that the EI-ELM’s training time is too long. In addition, EI-ELM can only add hidden nodes one by one. This paper proposes a fast method for EI-ELM (referred to as FI-ELM). At each learning step, several hidden nodes are randomly generated and the hidden nodes selected by the multiresponse sparse regression (MRSR) are added to the existing network. The output weights of the network are updated by a fast iterative method. The experimental results show that compared with EI-ELM, FI-ELM spends less time on training. Taking this advantage, FI-ELM can generate more hidden nodes to find the hidden node leading to larger residual error decreasing.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Zhou, G., Zhao, Q., Zhang, Y., Adalı, T., Xie, S., Cichocki, A.: Linked component analysis from matrices to high-order tensors: applications to biomedical data. Proc. IEEE 104(2), 310–331 (2016)

    Article  Google Scholar 

  2. Zhou, G., Cichocki, A., Zhang, Y., Mandic, D.P.: Group component analysis for multiblock data: common and individual feature extraction. IEEE Trans. Neural Netw. Learn. Syst. 27(11), 2426–2439 (2016)

    Article  MathSciNet  Google Scholar 

  3. Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17(4), 879–892 (2006)

    Article  Google Scholar 

  4. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70(1–3), 489–501 (2006)

    Article  Google Scholar 

  5. Lan, Y., Soh, Y., Huang, G.B.: Ensemble of online sequential extreme learning machine. Neurocomputing 72(13–15), 3391–3395 (2009)

    Article  Google Scholar 

  6. Lan, Y., Soh, Y., Huang, G.B.: Random search enhancement of error minimized extreme learning machine. In: European Symposium on Artificial Neural Networks, Esann 2010, Bruges (2010)

    Google Scholar 

  7. Huang, G.B., Chen, L.: Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18), 3460–3468 (2008)

    Article  Google Scholar 

  8. Zhang, R., Lan, Y., Huang, G.B., Xu, Z.B., Soh, Y.C.: Dynamic extreme learning machine and its approximation capability. IEEE Trans. Cybern. 43(6), 2054–2065 (2013)

    Article  Google Scholar 

  9. Huang, G.B., Chen, L.: Convex incremental extreme learning machine. Neurocomputing 70(16–18), 3056–3062 (2007)

    Article  Google Scholar 

  10. Rong, H.J., Ong, Y.S., Tan, A.H., Zhu, Z.X.: A fast pruned-extreme learning machine for classification problem. Neurocomputing 72(1–3), 359–366 (2008)

    Article  Google Scholar 

  11. Huang, G.B., Zhou, H., Ding, X., Zhang, R.: Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. B 42(2), 513–529 (2012)

    Article  Google Scholar 

  12. Wan, Y., Song, S., Huang, G.: Incremental extreme learning machine based on cascade neural networks. In: IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp. 1889–1894. IEEE (2015)

    Google Scholar 

  13. Similä, T., Tikka, J.: Multiresponse sparse regression with application to multidimensional scaling. In: Duch, W., Kacprzyk, J., Oja, E., Zadrożny, S. (eds.) ICANN 2005. LNCS, vol. 3697, pp. 97–102. Springer, Heidelberg (2005). doi:10.1007/11550907_16

    Google Scholar 

  14. Efron, B., Hastie, T., Johnstone, I., Tibshirani, R.: Least angle regression. Ann. Stat. 32(2), 407–499 (2004)

    Article  MATH  MathSciNet  Google Scholar 

  15. Zhang, F.: The Schur Complement and Its Applications. Numerical Methods and Algorithms, vol. 4. Springer, Boston (2005)

    Book  MATH  Google Scholar 

  16. UCI repository of machine learning databases. http://www.ics.uci.edu/mlearn/MLRepository.html

Download references

Acknowledgments

The work is supported by National Natural Science Foundation of China (61372142, U1401252), Fundamental Research Funds for the Central Universities SCUT (2017MS062), Guangzhou city science and technology research projects(201508010023, 201604016133).

Author information

Authors and Affiliations

  1. School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China

    Zhihui Lao, Zhiheng Zhou & Junchu Huang

Authors
  1. Zhihui Lao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zhiheng Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junchu Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhiheng Zhou .

Editor information

Editors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Derong Liu

  2. Guangdong University of Technology, Guangzhou, China

    Shengli Xie

  3. South China University of Technology, Guangzhou, China

    Yuanqing Li

  4. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Dongbin Zhao

  5. King Fahd University of Petroleum and Minerals, Dhahran, Saudi Arabia

    El-Sayed M. El-Alfy

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Lao, Z., Zhou, Z., Huang, J. (2017). Incremental Extreme Learning Machine via Fast Random Search Method. In: Liu, D., Xie, S., Li, Y., Zhao, D., El-Alfy, ES. (eds) Neural Information Processing. ICONIP 2017. Lecture Notes in Computer Science(), vol 10634. Springer, Cham. https://doi.org/10.1007/978-3-319-70087-8_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-70087-8_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-70086-1

  • Online ISBN: 978-3-319-70087-8

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation