Skip to main content

Advertisement

Springer Nature Link
Log in

Two-Stage Game Strategy for Multiclass Imbalanced Data Online Prediction

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

For multiclass imbalanced data online prediction, how to design a self-adapted model is a challenging problem. To address this issue, a novel dynamic multi-classification algorithm which uses two-stage game strategy has been put forward. Different from typical imbalanced classification methods, the proposed approach provided a self-updating model quantificationally, which can match the changes of arriving sample chunk automatically. In data generation phase, two dynamic ELMs with game theory are utilized for generating the lifelike minority class to equilibrate the distribution of different samples. In model update phase, both the current prediction performance and the cost sensitivity are taken into consideration simultaneously. According to the suffer loss and the shifty imbalance ratio, the proposed method develops the relationship between new weight and individual model, and an aggregate model of game theory is adopted to calculate the combination weight. These strategies help the algorithm reduce fitting error of sequence fragments. Also, alterative hidden-layer output matrix can be calculated according to the current fragment, thus building the steady network architecture in the next chunk. Numerical experiments are conducted on eight multiclass UCI datasets. The results demonstrate that the proposed algorithm not only has better generalization performance, but also improves the predictive ability of ELM method for minority samples.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Yang S, Zhang L, He L et al (2018) Sparse low-rank component-based representation for face recognition with low-quality images. IEEE Trans Inf Forensics Secur 14(1):251–261

    Google Scholar 

  2. Ko KE, Sim KB (2018) Deep convolutional framework for abnormal behavior detection in a smart surveillance system. Eng Appl Artif Intell 67:226–234

    Google Scholar 

  3. Barushka A, Hajek P (2018) Spam filtering using integrated distribution-based balancing approach and regularized deep neural networks. Appl Intell 48(10):3538–3556

    Google Scholar 

  4. Yuan X, Xie L, Abouelenien M (2018) A regularized ensemble framework of deep learning for cancer detection from multi-class, imbalanced training data. Pattern Recognit 77:160–172

    Google Scholar 

  5. He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284

    Google Scholar 

  6. Aurelio YS, de Almeida GM, de Castro CL et al (2019) Learning from imbalanced data sets with weighted cross-entropy function. Neural Process Lett 50(2):1937–1949

    Google Scholar 

  7. Zhang Q, Zhang P, Long G et al (2016) Online learning from trapezoidal data streams. IEEE Trans Knowl Data Eng 28(10):2709–2723

    MathSciNet  Google Scholar 

  8. Maurya CK, Toshniwal D, Venkoparao GV (2016) Online sparse class imbalance learning on big data. Neurocomputing 216:250–260

    Google Scholar 

  9. Chawla NV, Bowyer KW, Hall LO et al (2002) SMOTE: synthetic minority over-sampling technique. J Artif Intell Res 16:321–357

    MATH  Google Scholar 

  10. Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2012) DBSMOTE: density-based synthetic minority over-sampling technique. Appl Intell 36(3):664–684

    Google Scholar 

  11. Fernndez A, Garcia S, Herrera F et al (2018) SMOTE for learning from imbalanced data: progress and challenges, marking the 15-year anniversary. J Artif Intell Res 61:863–905

    MathSciNet  MATH  Google Scholar 

  12. Zong W, Huang GB, Chen Y (2013) Weighted extreme learning machine for imbalance learning. Neurocomputing 101:229–242

    Google Scholar 

  13. Huang G, Huang GB, Song S et al (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48

    MATH  Google Scholar 

  14. Yu H, Sun C, Yang X et al (2016) ODOC-ELM: optimal decision outputs compensation-based extreme learning machine for classifying imbalanced data. Knowl Based Syst 92:55–70

    Google Scholar 

  15. Liang NY, Huang GB, Saratchandran P et al (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423

    Google Scholar 

  16. Mirza B, Lin Z, Toh KA (2016) Weighted online sequential extreme learning machine for class imbalance learning. Neural Process Lett 38(3):465–486

    Google Scholar 

  17. Mirza B, Lin Z (2016) Meta-cognitive online sequential extreme learning machine for imbalanced and concept-drifting data classification. Neural Netw 80:79–94

    Google Scholar 

  18. Ahachad A, Omari A, Figueiras-Vidal AR (2015) Neighborhood guided smoothed emphasis for real adaboost ensembles. Neural Process Lett 42(1):155–165

    Google Scholar 

  19. Mirza B, Lin Z, Liu N (2015) Ensemble of subset online sequential extreme learning machine for class imbalance and concept drift. Neurocomputing 149:316–329

    Google Scholar 

  20. Li K, Kong X, Lu Z et al (2014) Boosting weighted ELM for imbalanced learning. Neurocomputing 128:15–21

    Google Scholar 

  21. Mao W, Wang J, He L et al (2017) Online sequential prediction of imbalance data with two-stage hybrid strategy by extreme learning machine. Neurocomputing 261:94–105

    Google Scholar 

  22. Wang S, Minku L L, Yao X (2013) A learning framework for online class imbalance learning. In: IEEE symposium on computational intelligence and ensemble learning (CIEL). IEEE, pp 36-45

  23. Sez JA, Krawczyk B, Woniak M (2016) Analyzing the oversampling of different classes and types of examples in multi-class imbalanced datasets. Pattern Recognit 57:164–178

    Google Scholar 

  24. Han H, Wang WY, Mao BH (2005) Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. International conference on intelligent computing. Springer, Berlin, pp 878–887

    Google Scholar 

  25. Bunkhumpornpat C, Sinapiromsaran K, Lursinsap C (2009) Safe-level-smote: Safe-level-synthetic minority over-sampling technique for handling the class imbalanced problem. Pacific-Asia conference on knowledge discovery and data mining. Springer, Berlin, pp 475–482

    Google Scholar 

  26. Tahir MA, Kittler J, Yan F (2012) Inverse random under sampling for class imbalance problem and its application to multi-label classification. Pattern Recognit 45(10):3738–3750

    Google Scholar 

  27. Gonzlez S, Garca S, Li ST et al (2019) Chain based sampling for monotonic imbalanced classification. Inf Sci 474:187–204

    Google Scholar 

  28. Pouyanfar S, Chen S C, Shyu M L (2018) Deep spatio-temporal representation learning for multi-class imbalanced data classification. In: IEEE international conference on information reuse and integration (IRI). IEEE, pp 386–393

  29. Guo Y, Xiao H (2018) Multiclass multiple kernel learning using hypersphere for pattern recognition. Appl Intell 48(9):2746–2754

    Google Scholar 

  30. Liu G, Yang Y, Li B (2018) Fuzzy rule-based oversampling technique for imbalanced and incomplete data learning. Knowl Based Syst 158:154–174

    Google Scholar 

  31. Abdi L, Hashemi S (2015) To combat multi-class imbalanced problems by means of over-sampling techniques. IEEE Trans Knowl Data Eng 28(1):238–251

    Google Scholar 

  32. Garca S, Zhang ZL, Altalhi A et al (2018) Dynamic ensemble selection for multi-class imbalanced datasets. Inf Sci 445:22–37

    MathSciNet  Google Scholar 

  33. Serafino F, Pio G, Ceci M (2018) Ensemble learning for multi-type classification in heterogeneous networks. IEEE Trans Knowl Data Eng 30(12):2326–2339

    Google Scholar 

  34. Lu J, Zhao P, Hoi SCH (2016) Online passive-aggressive active learning. Mach Learn 103(2):141–183

    MathSciNet  MATH  Google Scholar 

  35. Wang J, Zhao P, Hoi SCH et al (2013) Online feature selection and its applications. IEEE Trans Knowl Data Eng 26(3):698–710

    Google Scholar 

  36. Orabona F, Keshet J, Caputo B (2009) Bounded kernel-based online learning. J Mach Learn Res 10:2643–2666

    MathSciNet  MATH  Google Scholar 

  37. Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70(16–18):3056–3062

    Google Scholar 

  38. Feng G, Huang GB, Lin Q et al (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357

    Google Scholar 

  39. Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36(3):285–305

    Google Scholar 

  40. Bai Z, Huang GB, Wang D et al (2014) Sparse extreme learning machine for classification. IEEE Trans Cybern 44(10):1858–1870

    Google Scholar 

  41. Zhang R, Lan Y, Huang G et al (2012) Universal approximation of extreme learning machine with adaptive growth of hidden nodes. IEEE Trans Neural Netw Learn Syst 23(2):365–371

    Google Scholar 

  42. Zhang R, Lan Y, Huang GB et al (2013) Dynamic extreme learning machine and its approximation capability. IEEE Trans Cybern 43(6):2054–2065

    Google Scholar 

  43. Tang J, Deng C, Huang GB (2015) Extreme learning machine for multilayer perceptron. IEEE Trans Neural Netw Learn Syst 27(4):809–821

    MathSciNet  Google Scholar 

  44. Xue Y, Zhang L, Wang B et al (2018) Nonlinear feature selection using Gaussian kernel SVM-RFE for fault diagnosis. Appl Intell 48(10):3306–3331

    Google Scholar 

  45. Savitha R, Suresh S, Kim HJ (2014) A meta-cognitive learning algorithm for an extreme learning machine classifier. Cogn Comput 6(2):253–263

    Google Scholar 

  46. Goodfellow I, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial nets. In: Advances in neural information processing systems, pp 2672–2680

  47. Kroese DP, Porotsky S, Rubinstein RY (2006) The cross-entropy method for continuous multi-extremal optimization. Methodol Comput Appl Probab 8(3):383–407

    MathSciNet  MATH  Google Scholar 

  48. Frank A, Asuncion A. UCI Machine Learning Repository, Irvine. http://archive.ics.uci.edu/ml

  49. Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27(8):861–874

    MathSciNet  Google Scholar 

Download references

Acknowledgements

The work was supported by the National Natural Science Foundation of China under Grant 61775022 and U19A2063. Development Program of Science and Technology of Jilin Province of China (20180519012JH).We especially thank to Xiaoying Sun for his contribution to the paper. He supports the experiment of the paper and help modify the contribution.

Author information

Authors and Affiliations

  1. School of Computer Science and Technology, Changchun University of Science and Technology, Changchun, 130022, Jilin, China

    Haiyang Yu, Chunyi Chen & Huamin Yang

Authors
  1. Haiyang Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Chunyi Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huamin Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Haiyang Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yu, H., Chen, C. & Yang, H. Two-Stage Game Strategy for Multiclass Imbalanced Data Online Prediction. Neural Process Lett 52, 2493–2512 (2020). https://doi.org/10.1007/s11063-020-10358-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10358-w

Keywords

Search

Navigation