Skip to main content

Advertisement

Springer Nature Link
Log in

An experimental evaluation of extreme learning machines on several hardware devices

  • Extreme Learning Machine and Deep Learning Networks
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

As an important learning algorithm, extreme learning machine (ELM) is known for its excellent learning speed. With the expansion of ELM’s applications in the field of classification and regression, the need for its real-time performance is increasing. Although the use of hardware acceleration is an obvious solution, how to select the appropriate acceleration hardware for ELM-based applications is a topic worthy of further discussion. For this purpose, we designed and evaluated the optimized ELM algorithms on three kinds of state-of-the-art acceleration hardware, i.e., multi-core CPU, Graphics Processing Unit (GPU), and Field-Programmable Gate Array (FPGA) which are all suitable for matrix multiplication optimization. The experimental results showed that the speedup ratio of these optimized algorithms on acceleration hardware achieved 10–800. Therefore, we suggest that (1) use GPU to accelerate ELM algorithms for large dataset, and (2) use FPGA for small dataset because of its lower power, especially for some embedded applications. We also opened our source code.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://archive.ics.uci.edu/ml/datasets/Covertype.

  2. http://archive.ics.uci.edu/ml/datasets/Covertype.

  3. https://developer.nvidia.com.

  4. http://archive.ics.uci.edu/ml/datasets/Covertype.

  5. https://archive.ics.uci.edu/ml/machine-learning-databases/ionosphere/.

  6. https://archive.ics.uci.edu/ml/machine-learning-databases/undocumented/connectionist-bench/sonar/.

  7. https://github.com/bombe-org/ELM_exp.

  8. https://developer.nvidia.com/cublas.

  9. http://www.culatools.com/.

  10. https://github.com/bombe-org/ELM_GPU/.

References

  1. Alia-Martinez M, Antoñanzas J, Antonanzas-Torres F, Pernía-Espinoza A, Urraca R (2015) A straightforward implementation of a gpu-accelerated ELM in R with NVIDIA graphic cards. In: International conference on hybrid artificial intelligence systems. Springer, Berlin, pp 656–667

  2. Baldi P, Sadowski P, Whiteson D (2014) Searching for exotic particles in high-energy physics with deep learning. Nat Commun 5:4308

    Article  Google Scholar 

  3. Deng L, Yu D, et al (2014) Deep learning: methods and applications. Found Trends® Signal Process 7(3–4):197–387

  4. Ding L, Xin J, Wang G (2016) An efficient query processing optimization based on ELM in the cloud. Neural Comput Appl 27(1):35–44. https://doi.org/10.1007/s00521-013-1543-3

    Article  Google Scholar 

  5. Frances-Villora JV, Rosado-Muñoz A, Martínez-Villena JM, Bataller-Mompean M, Guerrero JF, Wegrzyn M (2016) Hardware implementation of real-time extreme learning machine in FPGA: analysis of precision, resource occupation and performance. Comput Electr Eng 51:139–156

    Article  Google Scholar 

  6. Hagan MT, Demuth HB, Beale MH, De Jesús O (1996) Neural network design, vol 20. Pws Pub, Boston

    Google Scholar 

  7. He Q, Du C, Wang Q, Zhuang F, Shi Z (2011) A parallel incremental extreme SVM classifier. Neurocomputing 74(16):2532–2540

    Article  Google Scholar 

  8. He Q, Shang T, Zhuang F, Shi Z (2013) Parallel extreme learning machine for regression based on mapreduce. Neurocomputing 102:52–58

    Article  Google Scholar 

  9. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Article  Google Scholar 

  10. Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460–3468

    Article  Google Scholar 

  11. Huang GB, Chen L, Siew CK et al (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892

    Article  Google Scholar 

  12. Huang GB, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74(1–3):155–163

    Article  Google Scholar 

  13. Huang GB, Liang NY, Rong HJ, Saratchandran P, Sundararajan N (2005) On-line sequential extreme learning machine. Comput Intell 2005:232–237

    Google Scholar 

  14. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2(2):107–122

    Article  Google Scholar 

  15. Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B (Cybernetics) 42(2):513–529

    Article  Google Scholar 

  16. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Article  Google Scholar 

  17. Jeowicz T, Gajdo P, Uher V, Snáel V (2015) Classification with extreme learning machine on GPU. In: 2015 international conference on intelligent networking and collaborative systems (INCOS), pp 116–122. IEEE

  18. Li H, Wu G (2014) Map matching for taxi GPS data with extreme learning machine. In: Advanced data mining and applications—10th international conference, ADMA 2014, Guilin, China, December 19–21, 2014. Proceedings, pp 447–460. https://doi.org/10.1007/978-3-319-14717-8_35

  19. Li J, Wang B, Wang G, Zhang Y (2016) Probabilistic threshold query optimization based on threshold classification using ELM for uncertain data. Neurocomputing 174:211–219. https://doi.org/10.1016/j.neucom.2015.05.122

    Article  Google Scholar 

  20. Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Article  Google Scholar 

  21. Liaw A, Wiener M et al (2002) Classification and regression by randomforest. R News 2(3):18–22

    Google Scholar 

  22. Ma Y, Yuan Y, Wang G, Bi X, Qin H (2018) Trust-aware personalized route query using extreme learning machine in location-based social networks. Cognit Comput 10(6):965–979. https://doi.org/10.1007/s12559-018-9600-y

    Article  Google Scholar 

  23. Magsi H, Sodhro AH, Chachar FA, Abro SAK, Sodhro GH, Pirbhulal S (2018) Evolution of 5g in internet of medical things. In: 2018 international conference on computing, mathematics and engineering technologies (iCoMET), pp 1–7. IEEE

  24. Rong HJ, Huang GB, Sundararajan N, Saratchandran P (2009) Online sequential fuzzy extreme learning machine for function approximation and classification problems. IEEE Trans Syst Man Cybern Part B (Cybernetics) 39(4):1067–1072

    Article  Google Scholar 

  25. Safaei A, Wu QJ, Yang Y, Akılan T (2017) System-on-a-chip (soc)-based hardware acceleration for extreme learning machine. In: 2017 24th IEEE international conference on electronics, circuits and systems (ICECS), pp 470–473. IEEE

  26. Schalkoff RJ (1997) Artificial neural networks, vol 1. McGraw-Hill, New York

    MATH  Google Scholar 

  27. Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117

    Article  Google Scholar 

  28. Sodhro AH, Luo Z, Sangaiah AK, Baik SW (2019) Mobile edge computing based QOS optimization in medical healthcare applications. Int J Inf Manag 45:308–318

    Article  Google Scholar 

  29. Sodhro AH, Malokani AS, Sodhro GH, Muzammal M, Zongwei L (2019) An adaptive QOS computation for medical data processing in intelligent healthcare applications. Neural Comput Appl, pp 1–12

  30. Sodhro AH, Pirbhulal S, de Albuquerque VHC (2019) Artificial intelligence-driven mechanism for edge computing-based industrial applications. IEEE Trans Ind Inf 15(7):4235–4243. https://doi.org/10.1109/TII.2019.2902878

    Article  Google Scholar 

  31. Sodhro AH, Pirbhulal S, Qaraqe M, Lohano S, Sodhro GH, Junejo NUR, Luo Z (2018) Power control algorithms for media transmission in remote healthcare systems. IEEE Access 6:42384–42393

    Article  Google Scholar 

  32. Sodhro AH, Pirbhulal S, Sodhro GH, Gurtov A, Muzammal M, Luo Z (2018) A joint transmission power control and duty-cycle approach for smart healthcare system. IEEE Sens J 19(19):8479–8486

    Article  Google Scholar 

  33. Sodhro AH, Shaikh FK, Pirbhulal S, Lodro MM, Shah MA (2017) Medical-QoS based telemedicine service selection using analytic hierarchy process. In: Handbook of large-scale distributed computing in smart healthcare. Springer, Berlin, pp 589–609

  34. Sun Y, Yuan Y, Wang G (2011) An OS-ELM based distributed ensemble classification framework in p2p networks. Neurocomputing 74(16):2438–2443

    Article  Google Scholar 

  35. Suykens JA, Vandewalle J (1999) Least squares support vector machine classifiers. Neural Process Lett 9(3):293–300

    Article  Google Scholar 

  36. Van Heeswijk M, Miche Y, Oja E, Lendasse A (2011) Gpu-accelerated and parallelized elm ensembles for large-scale regression. Neurocomputing 74(16):2430–2437

    Article  Google Scholar 

  37. Wang B, Wang G, Li J, Wang B (2012) Update strategy based on region classification using ELM for mobile object index. Soft Comput 16(9):1607–1615

    Article  Google Scholar 

  38. Wang G, Zhao Y, Wang D (2008) A protein secondary structure prediction framework based on the extreme learning machine. Neurocomputing 72(1–3):262–268

    Article  Google Scholar 

  39. Woods L, Teubner J, Alonso G (2011) Real-time pattern matching with FPGAD. In: 2011 IEEE 27th international conference on data engineering (ICDE), pp 1292–1295. IEEE

  40. Yeam TC, Ismail N, Mashiko K, Matsuzaki T (2017) FPGA implementation of extreme learning machine system for classification. In: Region 10 conference, TENCON 2017-2017 IEEE, pp 1868–1873. IEEE

  41. Zhang R, Huang GB, Sundararajan N, Saratchandran P (2007) Multicategory classification using an extreme learning machine for microarray gene expression cancer diagnosis. IEEE/ACM Trans Comput Biol Bioinform (TCBB) 4(3):485–495

    Article  Google Scholar 

  42. Zhang Z, Zhao X, Wang G, Bi X (2018) A new point-of-interest classification model with an extreme learning machine. Cognit Comput 10(6):951–964. https://doi.org/10.1007/s12559-018-9599-0

    Article  Google Scholar 

  43. Xg Zhao, Wang G, Bi X, Gong P, Zhao Y (2011) XML document classification based on ELM. Neurocomputing 74(16):2444–2451

    Article  Google Scholar 

  44. Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10):1759–1763

    Article  Google Scholar 

Download references

Acknowledgements

Gang Wu is supported by the NSFC (Grant No.61872072) and the State Key Laboratory of Computer Software New Technology Open Project Fund (Grant No.KFKT2018B05). Guoren Wang is the corresponding author of this paper. Guoren Wang is supported by the NSFC (Grant No. U1401256, 61732003, 61332006 and 61729201).

Author information

Authors and Affiliations

  1. Computer Science and Engineering, Northeastern University, Shenyang, China

    Liang Li & Gang Wu

  2. Computer Science and Technology, Beijing Institute of Technology, Beijing, China

    Guoren Wang & Qi Zhang

  3. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Gang Wu

Authors
  1. Liang Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Guoren Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Gang Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qi Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Guoren Wang.

Ethics declarations

Conflict of interest

The authors declared that they have no conflict of interest to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, L., Wang, G., Wu, G. et al. An experimental evaluation of extreme learning machines on several hardware devices. Neural Comput & Applic 32, 14385–14397 (2020). https://doi.org/10.1007/s00521-019-04481-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-019-04481-6

Keywords