Skip to main content

Advertisement

Springer Nature Link
Log in

On optimization based extreme learning machine in primal for regression and classification by functional iterative method

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

In this paper, the recently proposed extreme learning machine in the aspect of optimization method by Huang et al. (Neurocomputing, 74: 155–163, 2010) has been considered in its primal form whose solution is obtained by solving an absolute value equation problem by a simple, functional iterative algorithm. It has been proved under sufficient conditions that the algorithm converges linearly. The pseudo codes of the algorithm for regression and classification are given and they can be easily implemented in MATLAB. Experiments were performed on a number of real-world datasets using additive and radial basis function hidden nodes. Similar or better generalization performance of the proposed method in comparison to support vector machine (SVM), extreme learning machine (ELM), optimally pruned extreme learning machine (OP-ELM) and optimization based extreme learning machine (OB-ELM) methods with faster learning speed than SVM and OB-ELM demonstrates its effectiveness and usefulness.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Balasundaram S, Gupta D, Kapil (2014) 1-norm extreme learning machine for regression and multiclass classification using Newton method. Neuocomputing 128:4–14

    Article  Google Scholar 

  2. Balasundaram S, Kapil (2013) On extreme learning machine for ε–insensitive regression in the primal by Newton method. Neural Comput Appl 22:559–567

    Article  Google Scholar 

  3. Balasundaram S, Kapil (2011) Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives. Neurocomputing 74:2511–2519

    Article  Google Scholar 

  4. Binu PC, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161

    Article  Google Scholar 

  5. Box GEP, Jenkins GM (1976) Time series analysis: forecasting and Control. Holden-Day, San Francisco

    MATH  Google Scholar 

  6. Chen Z, Zhu H, Wang Y (2013) A modified extreme learning machine with sigmoidal activation functions. Neural Comput Appl 22:541–550

    Article  MathSciNet  Google Scholar 

  7. Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel based learning methods. Cambridge University Press, Cambridge

    Book  MATH  Google Scholar 

  8. DELVE (2005) Data for evaluating learning in valid experiments, http://www.cs.toronto.edu/~delve/data

  9. Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1-30

  10. Deng W, Chen L (2010) Color image watermarking using regularized extreme learning machine. Neural Netw World 20(3):317–330

    Google Scholar 

  11. Gretton A, Doucet A, Herbrich R, Rayner PJW, Scholkopf B (2001) Support vector regression for black-box system identification, In: Proceedings of the 11th IEEE Workshop on Statistical Signal Processing

  12. Feng G, Huang G-B, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357

    Article  Google Scholar 

  13. Frenay B, Verleysen M (2010) Using SVMs with randomized feature spaces: an extreme learning approach. In: Proceedings of the 18th European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, pp. 315–320

  14. Fu A, Dong C, Wang L (2014) An experimental study on stability and generalization of extreme learning machines. Int J Mach Learn Cybern. doi:10.1007/s13042-014-0238-0

    Google Scholar 

  15. Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062

    Article  Google Scholar 

  16. Huang G-B, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468

    Article  Google Scholar 

  17. Huang G-B, Chen L, Siew C-K (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  18. Huang G-B, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163

    Article  Google Scholar 

  19. Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122

    Article  Google Scholar 

  20. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst, Man Cybern Part B Cybern 42(2):513–528

    Article  Google Scholar 

  21. Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501

    Article  Google Scholar 

  22. Jun W, Shitong W, Chung F (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271

    Article  Google Scholar 

  23. Lan Y, Soh C, Huang G-B (2010) Two-stage extreme learning machine regression. Neurocomputing 73:3028–3038

    Article  Google Scholar 

  24. Liu Q, He Q, Shi Z (2008) Extreme support vector machine classifier. LNCS 5012:222–233

    Google Scholar 

  25. Mangasarian OL (2002) A finite newton method for classification. Optim Methods Softw 17:913–929

    Article  MathSciNet  MATH  Google Scholar 

  26. Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162

    Article  Google Scholar 

  27. Minhas R, Baradarani A, Seifzadeh S, Wu QMJ (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917

    Article  Google Scholar 

  28. Mohammed AA, Wu QMJ, Sid-Ahmed MA (2010) Application of wave atoms decomposition and extreme learning machine for finger print classification. LNCS 6112:246–256

    Google Scholar 

  29. Mukherjee S, Osuna E, Girosi F (1997) Nonlinear prediction of chaotic time series using support vector machines, in: NNSP’97: Neural Networks for Signal Processing VII: in Proceedigs of IEEE Signal Processing Society Workshop, Amelia Island, FL, USA, pp. 511–520

  30. Muller KR, Smola AJ, Ratsch G, Schölkopf B, Kohlmorgen J (1999) Using support vector machines for time series prediction. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 243–254

    Google Scholar 

  31. Murphy PM, Aha DW (1992) UCI repository of machine learning databases. University of California, Irvine. http://www.ics.uci.edu/~mlearn

  32. Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New York

    MATH  Google Scholar 

  33. Samet S, Miri A (2012) Privacy-preserving back-propagation and extreme learning machine algorithms. Data Knowl Eng 79–80:40–61

    Article  Google Scholar 

  34. Singh R, Balasundaram S (2007) Application of extreme learning machine for time series analysis. Int J Intell Technol 2:256–262

    Google Scholar 

  35. Sjoberg J, Zhang Q, Ljung L, Berveniste A, Delyon B, Glorennec P, Hjalmarsson H, Juditsky (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31:1691–1724

    Article  MathSciNet  MATH  Google Scholar 

  36. Souza LGM, Barreto GA (2006) Nonlinear system identification using local ARX models based on the self-organizing map. Learning and Nonlinear Models-Revista da Sociedade Brasileira de Redes Neurals (SBRN) 4(2):112–123

    Article  Google Scholar 

  37. Vapnik VN (2000) The nature of statistical learning theory, 2nd edn. Springer, New York

    Book  MATH  Google Scholar 

  38. Wang X, Shao Q, Qing M, Zhai J (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9

    Article  Google Scholar 

  39. Wang X, Chen A, Feng H (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525

    Article  Google Scholar 

  40. Wang XX, Chen S, Lowe D, Harris CJ (2006) Sparse support vector regression based on orthogonal forward selection for generalized kernel model. Neurocomputing 70:462–474

    Article  Google Scholar 

  41. Yuan Y, Wang Y, Cao F (2011) Optimization approximation solution for regression problem based on extreme learning machine. Neurocomputing 74:2475–2482

    Article  Google Scholar 

  42. Zhai J, Xu H, Wang X (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502

    Article  Google Scholar 

  43. Zhou GL, Toh KC (2005) Super linear convergence of a newton-type algorithm for monotone equations. J Optim Theory Appl 125:205–221

    Article  MathSciNet  MATH  Google Scholar 

  44. Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recogn 38:1759–1763

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors are extremely thankful to the learned referees for their constructive comments that greatly improved the earlier version of the paper.

Author information

Authors and Affiliations

  1. School of Computer and Systems Sciences, Jawaharlal Nehru University, New Delhi, 110067, India

    S. Balasundaram & Deepak Gupta

Authors
  1. S. Balasundaram
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Deepak Gupta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Balasundaram.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Balasundaram, S., Gupta, D. On optimization based extreme learning machine in primal for regression and classification by functional iterative method. Int. J. Mach. Learn. & Cyber. 7, 707–728 (2016). https://doi.org/10.1007/s13042-014-0283-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-014-0283-8

Keywords