Skip to main content
SpringerLink
Log in

Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Extreme learning machine(ELM) is a simple and fast machine learning algorithm. However, similar to other conventional learning algorithms, the classical ELM can not well process the problem of imbalanced data distribution. In this paper, in order to improve the learning performance of classical ELM for imbalanced data learning, we present a novel variant of the ELM algorithm based on a hybrid cost function which employs the probability that given training sample belong in each class to calculate the G-mean. We perform comparable experiments for our approach and the state-of-the-arts methods on standard classification datasets which consist of 58 binary datasets and 9 multiclass datasets under different degrees of imbalance ratio. Experimental results show that our proposed algorithm can improve the classification performance significantly compared with other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Huang G-B, Zhu Q-Y, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501

    Google Scholar 

  2. Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans. Syst. Man Cybern. Part B (Cybern.) 42(2):513–529

    Google Scholar 

  3. Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Google Scholar 

  4. Huang G-B, Wang D, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybernet 2(2):107–122

    Google Scholar 

  5. Rumelhart DE, Hinton GE, Williams RJ (1988) Learning representations by back-propagating errors. In: Neurocomputing: foundations of research. MIT Press, Cambridge, MA, USA, pp 696–699

  6. Hagan MT, Menhaj MB (1994) Training feedforward networks with the marquardt algorithm. IEEE Trans Neural Netw 5(6):989–993

    Google Scholar 

  7. Zhi-Wang, Wang X-Z (09 2017) A deep stochastic weight assignment network and its application to chess playing. J Parallel Distrib Comput 117

  8. Xia M, Zhang Y, Weng L, Ye X (2012) Fashion retailing forecasting based on extreme learning machine with adaptive metrics of inputs. Knowl-Based Syst 36:253–259

    Google Scholar 

  9. Chen FL, Ou TY (2011) Sales forecasting system based on gray extreme learning machine with Taguchi method in retail industry. Expert Syst Appl 38(3):1336–1345

    Google Scholar 

  10. Samat A, Du P, Liu S, Li J, Cheng L (2014) \({{E}^{2}}{LMs}\) : ensemble extreme learning machines for hyperspectral image classification. IEEE J Selected Topics Appl Earth Obs Remote Sens 7(4):1060–1069

    Google Scholar 

  11. Zhang H, Li M (2014) Rwo-sampling: a random walk over-sampling approach to imbalanced data classification. Inf Fusion 20:99–116

    Google Scholar 

  12. Charte F, Rivera AJ, del Jesus MJ, Herrera F (2015) MLSMOTE: approaching imbalanced multilabel learning through synthetic instance generation. Knowl-Based Syst 89:385–397

    Google Scholar 

  13. Chawla NV, Bowyer KW, Hall LO, Kegelmeyer WP (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    MATH  Google Scholar 

  14. López V, del Río S, Benítez JM, Herrera F (2015) Cost-sensitive linguistic fuzzy rule based classification systems under the mapreduce framework for imbalanced big data. Fuzzy Sets Syst 258(1):5–38

    MathSciNet  Google Scholar 

  15. Yu H, Mu C, Sun C, Yang W, Yang X, Zuo X (2015) Support vector machine-based optimized decision threshold adjustment strategy for classifying imbalanced data. Knowl-Based Syst 76:67–78

    Google Scholar 

  16. Krawczyk B, Woźniak M, Schaefer G (2014) Cost-sensitive decision tree ensembles for effective imbalanced classification. Appl Soft Comput 14(Part C):554–562

    Google Scholar 

  17. Zong W, Huang G-B, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101(3):229–242

    Google Scholar 

  18. Toh K-A (2008) Deterministic neural classification. Neural Comput 20(6):1565–1595

    MathSciNet  MATH  Google Scholar 

  19. Deng W, Zheng Q, Chen L (April 2009) Regularized extreme learning machine. In: Proceedings of the IEEE symposium on computational intelligence and data mining, Nashville, TN, USA. IEEE, pp 389–395

  20. Li K, Kong X, Lu Z, Wenyin L, Yin J (2014) Boosting weighted ELM for imbalanced learning. Neurocomputing 128(5):15–21

    Google Scholar 

  21. Ri J-H, Liu L, Liu Y, Wu H-F, Huang W-L, Kim H (2018) Optimal weighted extreme learning machine for imbalanced learning with differential evolution. IEEE Comput Intell Mag 13(3):32–47

    Google Scholar 

  22. Yu H, Sun C, Yang X, Yang W, Shen J, Qi Y (2016) ODOC-ELM: optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data. Knowl-Based Syst 92(15):55–70

    Google Scholar 

  23. Mao W, Wang J, Xue Z (2017) An elm-based model with sparse-weighting strategy for sequential data imbalance problem. Int J Mach Learn Cybern 8(4):1333–1345

    Google Scholar 

  24. Mao W, He L, Yan Y, Wang J (2017) Online sequential prediction of bearings imbalanced fault diagnosis by extreme learning machine. Mech Syst Signal Process 83:450–473

    Google Scholar 

  25. Du J, Vong C-M, Pun C-M, Wong P-K, Ip W-F (2017) Post-boosting of classification boundary for imbalanced data using geometric mean. Neural Netw 96:101–114

    MATH  Google Scholar 

  26. Vong C-M, Du J, Wong C-M, Cao J-W (May 2018) Postboosting using extended g-mean for online sequential multiclass imbalance learning. IEEE Trans Neural Netw Learn Syst

  27. Cao W, Wang X, Ming Z, Gao J (2018) A review on neural networks with random weights. Neurocomputing 275:278–287

    Google Scholar 

  28. Wang X, Cao W (2018) Non-iterative approaches in training feed-forward neural networks and their applications. Soft Comput 22(11):34733476

    Google Scholar 

  29. Wang X-Z, Zhang T, Wang R (2017) Noniterative deep learning: incorporating restricted boltzmann machine into multilayer random weight neural networks. IEEE Trans Syst Man Cybern Syst

  30. Wang D, Li M (2017) Stochastic configuration networks: fundamentals and algorithms. IEEE Trans Cybern 47(10):3466–3479

    Google Scholar 

  31. Fletcher R (1981) Practical methods of optimization, volume 2: constrained optimization. Wiley, New York

    MATH  Google Scholar 

  32. Chawla NV (2010) Data mining for imbalanced datasets: an overview. In: Data mining and knowledge discovery handbook, 2nd edn. Springer, pp 875–886

  33. Schraudolph NN, Yu J, Gnter S (October 2007) A stochastic quasi-newton method for online convex optimization. In: Proceedings of 11th international conference on artificial intelligence and statistics, Vilamoura, Algarve, Portugal, pp 436–443

  34. Mokhtari A, Ribeiro A (2014) RES: regularized stochastic bfgs algorithm. IEEE Trans Signal Process 62(23):6089–6104

    MathSciNet  MATH  Google Scholar 

  35. Morales JL, Nocedal J (2011) Remark on algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound constrained optimization. ACM Trans Math Softw (TOMS) 38(1):7

    MATH  Google Scholar 

  36. Byrd RH, Lu P, Nocedal J, Zhu C (1995) A limited memory algorithm for bound constrained optimization. SIAM J Sci Comput 16(5):1190–1208

    MathSciNet  MATH  Google Scholar 

  37. Zhu C, Byrd RH, Lu P, Nocedal J (1997) Algorithm 778: L-bfgs-b: Fortran subroutines for large-scale bound-constrained optimization. ACM Trans Math Softw (TOMS) 23(4):550–560

    MathSciNet  MATH  Google Scholar 

  38. Mokhtari A, Ribeiro A (2015) Global convergence of online limited memory bfgs. J Mach Learn Res 16(1):3151–3181

    MathSciNet  MATH  Google Scholar 

  39. Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    MATH  Google Scholar 

  40. Alcalá-Fdez J, Fernández A, Luengo J, Derrac J, García S (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. Multiple-Valued Logic Soft Comput 17(2–3):255–287

    Google Scholar 

  41. Garcia S, Fernandez A, Luengo J, Herrera F (2009) A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability. Soft Comput 13(10):959–977

    Google Scholar 

  42. Zhou Z-H (2016) Machine learning. Qinghua University Press, Beijing

    Google Scholar 

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under Grant 61836015, 61771193.

Author information

Author notes

  1. Jong-Hyok Ri and Guanzhong Tian have contributed equally to this work.

Authors and Affiliations

  1. Institute of Cyber-Systems and Control, Zhejiang University, Hangzhou, 310027, China

    Jong-Hyok Ri, Guanzhong Tian, Yong Liu, Wei-hua Xu & Jun-gang Lou

  2. Institute of Information Technology, Kim Il Sung University, Pyongyang, 190016, Democratic People’s Republic of Korea

    Jong-Hyok Ri

  3. State Key Laboratory of Industrial Control Technology, Zhejiang University, Hangzhou, China

    Yong Liu & Wei-hua Xu

Authors
  1. Jong-Hyok Ri
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guanzhong Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yong Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wei-hua Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jun-gang Lou
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Liu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ri, JH., Tian, G., Liu, Y. et al. Extreme learning machine with hybrid cost function of G-mean and probability for imbalance learning. Int. J. Mach. Learn. & Cyber. 11, 2007–2020 (2020). https://doi.org/10.1007/s13042-020-01090-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-020-01090-x

Keywords

Search

Navigation