Skip to main content
SpringerLink
Log in

Extreme learning machines: new trends and applications

极限学习机: 新趋势与新应用

  • Review
  • Special Focus on High-Speed Signal Processing
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Extreme learning machine (ELM), as a new learning framework, draws increasing attractions in the areas of large-scale computing, high-speed signal processing, artificial intelligence, and so on. ELM aims to break the barriers between the conventional artificial learning techniques and biological learning mechanism and represents a suite of machine learning techniques in which hidden neurons need not to be tuned. ELM theories and algorithms argue that “random hidden neurons” capture the essence of some brain learning mechanisms as well as the intuitive sense that the efficiency of brain learning need not rely on computing power of neurons. Thus, compared with traditional neural networks and support vector machine, ELM offers significant advantages such as fast learning speed, ease of implementation, and minimal human intervention. Due to its remarkable generalization performance and implementation efficiency, ELM has been applied in various applications. In this paper, we first provide an overview of newly derived ELM theories and approaches. On the other hand, with the ongoing development of multilayer feature representation, some new trends on ELM-based hierarchical learning are discussed. Moreover, we also present several interesting ELM applications to showcase the practical advances on this subject.

摘要

创新点

极限学习机作为一种全新的机器学习理论和框架, 在大数据计算、 高速信号处理, 人工智能等领域越来越受到关注。 极限学习机旨在打破传统学习理论和生物学习机制之间的壁垒, 该理论认为人脑的学习效率不依赖于单个神经元的计算能力, 因此极限学习机通过随机产生的隐层神经元来逼近大脑学习机理, 取得了比传统神经网络和支持向量机更高的学习精度、 更快的训练速度以及更少的人为干预。 本文对近年来提出的极限学习机新理论和新方法进行综述, 在此基础上重点介绍基于极限学习机的多层特征表征方面的最新研究成果, 最后介绍极限学习机的实际应用。

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Rumelhart D E, Hinton G E, Williams R J. Learning representations by back-propagating errors. Nature, 1986, 323: 533–536

    Article  Google Scholar 

  2. Hagan M T, Menhaj M B. Training feedforward networks with the marquardt algorithm. IEEE Trans Neural Netw, 1994, 5: 989–993

    Article  Google Scholar 

  3. Wilamowski B M, Yu H. Neural network learning without backpropagation. IEEE Trans Neural Netw, 2010, 21: 1793–1803

    Article  Google Scholar 

  4. Chen S, Cowan C, Grant P. Orthogonal least squares learning algorithm for radial basis function networks. IEEE Trans Neural Netw, 1991, 2: 302–309

    Article  Google Scholar 

  5. Li K, Peng J X, Irwin G W. A fast nonlinear model identification method. IEEE Trans Automat Contr, 2005, 50: 1211–1216

    Article  MathSciNet  Google Scholar 

  6. Hornik K. Approximation capabilities of multilayer feedforward networks. Neural netw, 1991, 4: 251–257

    Article  Google Scholar 

  7. Hassoun M H. Fundamentals of Artificial Neural Networks. MIT Press, 1995. 35–55

    Google Scholar 

  8. Cybenko G. Approximation by superpositions of a sigmoidal function. Math Control Signal Syst, 1989, 2: 303–314

    Article  MATH  MathSciNet  Google Scholar 

  9. White H. Artificial Neural Networks: Approximation and Learning Theory. Blackwell Publishers, Inc., 1992. 30–100

    Google Scholar 

  10. Huang G B, Zhu Q Y, Siew C K. Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of IEEE International Joint Conference on Neural Networks, Budapest, 2004. 985–990

    Google Scholar 

  11. Huang G B, Zhou H, Ding X. Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B, 2012, 42: 513–529

    Article  Google Scholar 

  12. Huang G B. An insight into extreme learning machines: random neurons, random features and kernels. Cognitive Comput, 2014, 6: 376–390

    Article  Google Scholar 

  13. Suykens J A, Vandewalle J. Least squares support vector machine classifiers. Neural Process Lett, 1999, 9: 293–300

    Article  MathSciNet  Google Scholar 

  14. Huang G B, Chen L, Siew C K. Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw, 2006, 17: 879–892

    Article  Google Scholar 

  15. Huang G B, Chen L. Convex incremental extreme learning machine. Neurocomputing, 2007, 70: 3056–3062

    Article  Google Scholar 

  16. Huang G B, Chen L. Enhanced random search based incremental extreme learning machine. Neurocomputing, 2008, 71: 3460–3468

    Article  Google Scholar 

  17. Tang J, Deng C, Huang G B, et al. Compressed-domain ship detection on spaceborne optical image using deep neural network and extreme learning machine. IEEE Trans Geosci Remot Sen, 2014, in press

    Google Scholar 

  18. An L, Bhanu B. Image super-resolution by extreme learning machine. In: Proceedings of IEEE International Conference on Image Processing (ICIP), Orlando, 2012. 2209–2212

    Google Scholar 

  19. Suresh S, Venkatesh Babu R, Kim H J. No-reference image quality assessment using modified extreme learning machine classifier. Appl Soft Comput, 2009, 9: 541–552

    Article  Google Scholar 

  20. Decherchi S, Gastaldo P, Zunino R. Circular-ELM for the reduced-reference assessment of perceived image quality. Neurocomputing, 2013, 102: 78–89

    Article  Google Scholar 

  21. Minhas R, Baradarani A, Seifzadeh S. Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing, 2010, 73: 1906–1917

    Article  Google Scholar 

  22. Huang G B, Chen L. Convex incremental extreme learning machine. Neurocomputing, 2007, 70: 3056–3062

    Article  Google Scholar 

  23. Liang N Y, Huang G B, Saratchandran P. A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw, 2006, 17: 1411–1423

    Article  Google Scholar 

  24. Rong H J, Huang G B, Sundararajan N. Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern B, 2009, 39: 1067–1072

    Article  Google Scholar 

  25. Huang G, Song S, Gupta J N. Semi-supervised and unsupervised extreme learning machines. IEEE Trans Syst Man Cybern B, 2014, in press

    Google Scholar 

  26. Kasun L L, Zhou H, Huang G B, et al. Representational learning with extreme learning machine for big data. IEEE Intell Syst, 2013, 28: 31–34

    Article  Google Scholar 

  27. Tanabe K. Projection method for solving a singular system of linear equations and its applications. Num Math, 1971, 17: 203–214

    Article  MATH  MathSciNet  Google Scholar 

  28. Hoerl A E, Kennard R W. Ridge regression: biased estimation for nonorthogonal problems. Technometrics, 1970, 12: 55–67

    Article  MATH  Google Scholar 

  29. van Gestel T, de Brabanter J, de Moor B. Least Squares Support Vector Machines. Singapore: World Scientific, 2002

    Google Scholar 

  30. Anderson W N, Morley T D. Eigenvalues of the Laplacian of a graph. Linear Multilinear Algebra, 1985, 18: 141–145

    Article  MATH  MathSciNet  Google Scholar 

  31. Tang J, Deng C, Huang G B. A fast learning algorithm for multi-layer extreme learning machine. In: Proceedings of IEEE International Conference on Image Processing, 2014, (in press)

    Google Scholar 

  32. Beck A, Teboulle M. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J Imag Sci, 2009, 2: 183–202

    Article  MATH  MathSciNet  Google Scholar 

  33. Beck A, Teboulle M. Fast gradient-based algorithms for constrained total variation image denoising and deblurring problems. IEEE Trans Image Process, 2009, 18: 2419–2434

    Article  MathSciNet  Google Scholar 

  34. Hinton G, Osindero S, Teh Y W. A fast learning algorithm for deep belief nets. Neural comput, 2006, 18: 1527–1554

    Article  MATH  MathSciNet  Google Scholar 

  35. Hinton G E, Salakhutdinov R R. Reducing the dimensionality of data with neural networks. Science, 2006, 313: 504–507

    Article  MATH  MathSciNet  Google Scholar 

  36. Bengio Y. Learning deep architectures for AI. Found Trends Mach Learn, 2009, 2: 1–127

    Article  MATH  Google Scholar 

  37. Bengio Y, Courville A, Vincent P. Representation learning: a review and new perspectives. IEEE Trans Patt Anal Mach Intell, 2013, 35: 1798–1828

    Article  Google Scholar 

  38. Vincent P, Larochelle H, Bengio Y. Extracting and composing robust features with denoising autoencoders. In: Proceedings of ACM International Conference on Machine Learning, Helsinki, 2008. 1096–1103

    Google Scholar 

  39. Salakhutdinov R, Hinton G E. Deep Boltzmann machines. In: Proceedings of IEEE International Conference on Artificial Intelligence and Statistics, Clearwater Beach, 2009. 448–455

    Google Scholar 

  40. Zhu C, Zhou H, Wang R, et al. A novel hierarchical method of ship detection from spaceborne optical image based on shape and texture features. IEEE Trans Geosci Remot Sen, 2010, 48: 3446–3456

    Article  Google Scholar 

  41. Bi F, Liu F, Gao L. A hierarchical salient-region based algorithm for ship detection in remote sensing images. In: Zeng Z G, Wang J, eds. Advances in Neural Network Research and Applications. Berlin/Heidelberg: Springer, 2010. 729–738

    Chapter  Google Scholar 

  42. Wang Z, Bovik A C, Sheikh H R, et al. Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process, 2004, 13: 600–612

    Article  Google Scholar 

  43. Lee D D, Seung H S. Algorithms for non-negative matrix factorization. In: Leen T K, Dietterich T G, Tresp V, eds. Advances in Neural Information Processing Systems. Cambridge: MIT Press, 2001. 556–562

    Google Scholar 

  44. Lee D D, Seung H S. Learning the parts of objects by non-negative matrix factorization. Nature, 1999, 401: 788–791

    Article  Google Scholar 

  45. Buffalo E A, Fries P, Landman R, et al. A backward progression of attentional effects in the ventral stream. Proc Nat Acad Sci, 2010, 107: 361–365

    Article  Google Scholar 

  46. Motter B C. Focal attention produces spatially selective processing in visual cortical areas V1, V2, and V4 in the presence of competing stimuli. J Neurophysiol, 1993, 70: 909–909

    Google Scholar 

  47. Bouzerdoum A, Havstad A, Beghdadi A. Image quality assessment using a neural network approach. In: Proceedings of IEEE International Symposium on Signal Processing and Information Technology, Rome, 2004. 330–333

    Google Scholar 

  48. Carrai P, Heynderickz I, Gastaldo P, et al. Image quality assessment by using neural networks. In: Proceedings of IEEE International Symposium on Circuits and Systems, Scottsdale, 2002. V-253–V-256

    Google Scholar 

  49. Narwaria M, Lin W, Cetin A E. Scalable image quality assessment with 2D mel-cepstrum and machine learning approach. Patt Recog, 2012, 45: 299–313

    Article  Google Scholar 

  50. Narwaria M, Lin W. SVD-based quality metric for image and video using machine learning. IEEE Trans Syst Man Cybern B, 2012, 42: 347–364

    Article  Google Scholar 

  51. Liu T J, Lin W, Kuo C C J. Image quality assessment using multi-method fusion. IEEE Trans Image Process, 2013, 22: 1793–1807

    Article  MathSciNet  Google Scholar 

  52. Wang S, Deng C, Lin W, et al. A novel NMF-based image quality assessment metric using extreme learning machine. In: Proceedings of IEEE China Summit & International Conference on Signal and Information Processing, Beijing, 2013. 255–258

    Google Scholar 

  53. Zhu J, Wang N. Image quality assessment by visual gradient similarity. IEEE Trans Image Process, 2012, 21: 919–933

    Article  MathSciNet  Google Scholar 

  54. Wu J, Lin W, Shi G, et al. Perceptual quality metric with internal generative mechanism. IEEE Trans Image Process, 2013, 22: 43–54

    Article  MathSciNet  Google Scholar 

  55. Zhang L, Zhang D, Mou X. FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process, 2011, 20: 2378–2386

    Article  MathSciNet  Google Scholar 

  56. Li S, Zhang F, Ma L, et al. Image quality assessment by separately evaluating detail losses and additive impairments. IEEE Trans Multimedia, 2011, 13: 935–949

    Article  Google Scholar 

  57. Liu A, Lin W, Narwaria M. Image quality assessment based on gradient similarity. IEEE Trans Image Process, 2012, 21: 1500–1512

    Article  MathSciNet  Google Scholar 

  58. Wang Z, Li Q. Information content weighting for perceptual image quality assessment. IEEE Trans Image Process, 2011, 20: 1185–1198

    Article  MathSciNet  Google Scholar 

  59. Ross D A, Lim J, Lin R S, et al. Incremental learning for robust visual tracking. Int J Comput Vis, 2008, 77: 125–141

    Article  Google Scholar 

  60. Tang J, Deng C, Huang G B. Extreme learning machine for multilayer perceptron. IEEE Trans Neural Netw Learn Syst, 2014, in press

    Google Scholar 

  61. Zhang K, Zhang L, Yang M H. Real-time compressive tracking. In: Proceedings of European Conference on Computer Vision, Florence, 2012. 864–877

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Information and Electronics, Beijing Institute of Technology, Beijing, 100081, China

    ChenWei Deng, Jia Xu & JieXiong Tang

  2. School of Electronic Engineering, Nanyang Technological University, Singapore, Singapore

    GuangBin Huang

Authors
  1. ChenWei Deng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. GuangBin Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jia Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. JieXiong Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to GuangBin Huang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Deng, C., Huang, G., Xu, J. et al. Extreme learning machines: new trends and applications. Sci. China Inf. Sci. 58, 1–16 (2015). https://doi.org/10.1007/s11432-014-5269-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11432-014-5269-3

Keywords

关键词

Search

Navigation