Skip to main content
SpringerLink
Log in

A Comparative Study of Harmony Search Algorithms for Improving the Training Process of Extreme Learning Machines

  • Conference paper
  • First Online:
Book cover Advances in Soft Computing (MICAI 2017)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10632))

Included in the following conference series:

  • 386 Accesses

Abstract

Extreme Learning Machine (ELM) is a model for the training of single-hidden feedforward networks and is used extensively in problems of classification and regression for its good results and reduced training time compared to backpropagation. Previous work has shown that to define randomly the input weights and biases of the hidden layer reduces quality (accuracy) of ELM. This paper makes a comparative study of three variants of harmony search (the original Harmony Search, Global-best Harmony Search, and New Global Harmony Search), a memetic algorithm from the state of the art denominated M-ELM and a random walk algorithm named RW-ELM on 20 classical classification datasets available in the UCI repository. The results show that the best algorithm for training ELMs is Harmony Search and that the other two variants of this algorithm are better than M-ELM and RW-ELM when cross-validation is used. The experiments were performed at first using separate archives for training and testing, then using cross-validation with 5 folders. Based on literature, authors recommend the use of cross-validation technique because more realistic accuracy results can be obtained.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 74.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by back-propagating errors. Nature 323, 533–536 (1986)

    Article  Google Scholar 

  2. Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: theory and applications. Neurocomputing 70, 489–501 (2006)

    Article  Google Scholar 

  3. van Heeswijk, M.: Advances in Extreme Learning Machines (2015). http://urn.fi/URN:ISBN:978-952-60-6149-8

  4. Zhu, Q.Y., Qin, A.K., Suganthan, P.N., Huang, G.: Bin: evolutionary extreme learning machine. Pattern Recognit. 38, 1759–1763 (2005)

    Article  Google Scholar 

  5. Han, F., Yao, H.F., Ling, Q.H.: An improved evolutionary extreme learning machine based on particle swarm optimization. Neurocomputing 116, 87–93 (2013)

    Article  Google Scholar 

  6. Saraswathi, S., Sundaram, S., Sundararajan, N., Zimmermann, M., Nilsen-Hamilton, M.: ICGA-PSO-ELM approach for accurate multiclass cancer classification resulting in reduced gene sets in which genes encoding secreted proteins are highly represented. IEEE/ACM Trans. Comput. Biol. Bioinform. 8, 452–463 (2011)

    Article  Google Scholar 

  7. Silva, D.N.G., Pacifico, L.D.S., Ludermir, T.B.: An evolutionary extreme learning machine based on group search optimization. In: 2011 IEEE Congress of Evolutionary Computation (CEC), pp. 574–580 (2011)

    Google Scholar 

  8. Cao, J., Lin, Z., Huang, G.: Bin: self-adaptive evolutionary extreme learning machine. Neural Process. Lett. 36, 285–305 (2012)

    Article  Google Scholar 

  9. Valença, I., Valença, M.: Optimizing the extreme learning machine using harmony search for hydrologic time series forecasting. Intell. Data Eng. Autom. Learn. IDEAL 2012, 261–269 (2012)

    Google Scholar 

  10. Kong, H.: Evolving extreme learning machine paradigm with adaptive operator selection and parameter control. Int. J. Uncertain. Fuzziness Knowl. Base Syst. 21, 143–154 (2013)

    Article  MathSciNet  Google Scholar 

  11. Learning, A.E., Elm, M.: Evolutionary extreme learning machine based on particle swarm optimization and clustering strategies. In: Proceedings of the 2013 International Joint Conference Neural Networks, vol. 560, pp. 305–310 (2013)

    Google Scholar 

  12. Bazi, Y., Alajlan, N., Melgani, F., AlHichri, H., Malek, S., Yager, R.R.: Differential evolution extreme learning machine for the classification of hyperspectral images. IEEE Geosci. Remote Sens. Lett. 11, 1066–1070 (2014)

    Article  Google Scholar 

  13. Dash, R., Dash, P.K., Bisoi, R.: A self adaptive differential harmony search based optimized extreme learning machine for financial time series prediction. Swarm Evol. Comput. 19, 25–42 (2014)

    Article  Google Scholar 

  14. Liu, N., Wang, H., Member, S.: Ensemble based extreme learning machine. IEEE Sig. Process. Lett. 17, 754–757 (2010)

    Article  Google Scholar 

  15. Cao, J., Lin, Z., Huang, G., Liu, N.: Voting based extreme learning machine. Inf. Sci. (Ny) 185, 66–77 (2012)

    Article  MathSciNet  Google Scholar 

  16. Zhang, Y., Wu, J., Cai, Z., Zhang, P., Chen, L.: Memetic extreme learning machine. Pattern Recogn. 58, 135–148 (2016)

    Article  Google Scholar 

  17. Wolpert, D.H., Macready, W.G.: No free lunch theorems for optimization. IEEE Trans. Evol. Comput. 1, 67–82 (1997)

    Article  Google Scholar 

  18. Wong, W.K., Guo, Z.X.: A hybrid intelligent model for medium-term sales forecasting in fashion retail supply chains using extreme learning machine and harmony search algorithm. Int. J. Prod. Econ. 128, 614–624 (2010)

    Article  Google Scholar 

  19. Huang, G., Zhu, Q., Siew, C.: Extreme learning machine: a new learning scheme of feedforward neural networks. Neurocomputing 70, 489–501 (2006)

    Article  Google Scholar 

  20. Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans. Neural Netw. 17, 879–892 (2006)

    Article  Google Scholar 

  21. Geem, Z.W., Kim, J.H., Loganathan, G.V.: A new heuristic optimization algorithm: harmony search. Simulation 76, 60–68 (2001)

    Article  Google Scholar 

  22. Mahdavia, M., Fesangharyb, M., Damangirb, E.: An improved harmony search algorithm for solving optimization problems. Appl. Math. Comput. 188, 1567–1579 (2007)

    MathSciNet  Google Scholar 

  23. Lee, K.S., Geem, Z.W.: A new meta-heuristic algorithm for continuous engineering optimization: harmony search theory and practice. Comput. Methods Appl. Mech. Eng. 194, 3902–3933 (2005)

    Article  Google Scholar 

  24. Manjarres, D., et al.: A survey on applications of the harmony search algorithm. Eng. Appl. Artif. Intell. 26, 1818–1831 (2013)

    Article  Google Scholar 

  25. Cobos, C., Pérez, J., Estupiñan, D.: A survey of harmony search. Av. en Sist. e Inform. 8, 67–80 (2011)

    MATH  Google Scholar 

  26. Omran, M.G.H., Mahdavi, M.: Global-best harmony search. Appl. Math. Comput. 198, 643–656 (2008)

    MathSciNet  MATH  Google Scholar 

  27. Zou, D., Gao, L., Wu, J., Li, S., Li, Y.: A novel global harmony search algorithm for reliability problems. Comput. Ind. Eng. 58, 307–316 (2010)

    Article  Google Scholar 

  28. Cobos, C., Estupiñán, D., Pérez, J.: GHS + LEM: global-best harmony search using learnable evolution models. Appl. Math. Comput. 218, 2558–2578 (2011)

    MathSciNet  MATH  Google Scholar 

  29. Eberhart, R., Kennedy, J.: A new optimizer using particle swarm theory. In: Proceedings of the Sixth International Symposium Micro Machine and Human Science, pp. 39–43 (1995)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Information Technology Research Group (GTI), Universidad del Cauca, Popayán, Colombia

    Daniel Pusil, Carlos Cobos & Martha Mendoza

Authors
  1. Daniel Pusil
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos Cobos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Martha Mendoza
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Cobos .

Editor information

Editors and Affiliations

  1. Universidad Autónoma del Estado de Hidalgo, Pachuca, Mexico

    Félix Castro

  2. INFOTEC Aguascalientes, Aguascalientes, Mexico

    Sabino Miranda-Jiménez

  3. Tecnológico de Monterrey, Atizapán de Zaragoza, Mexico

    Miguel González-Mendoza

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pusil, D., Cobos, C., Mendoza, M. (2018). A Comparative Study of Harmony Search Algorithms for Improving the Training Process of Extreme Learning Machines. In: Castro, F., Miranda-Jiménez, S., González-Mendoza, M. (eds) Advances in Soft Computing. MICAI 2017. Lecture Notes in Computer Science(), vol 10632. Springer, Cham. https://doi.org/10.1007/978-3-030-02837-4_20

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-02837-4_20

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-02836-7

  • Online ISBN: 978-3-030-02837-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation