Skip to main content
SpringerLink
Log in

Hierarchical Ensemble Based Imbalance Classification

  • Conference paper
  • First Online:
Computational Science – ICCS 2022 (ICCS 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13350))

Included in the following conference series:

  • 1116 Accesses

Abstract

In this paper, we propose a hierarchical ensemble method for improved imbalance classification. Specifically, we perform the first-level ensemble based on bootstrap sampling with replacement to create an ensemble. Then, the second-level ensemble is generated based on two different weighting strategies, where the strategy having better performance is selected for the subsequent analysis. Next, the third-level ensemble is obtained via the combination of two methods for obtaining mean and covariance of multivariate Gaussian distribution, where the oversampling is then realized via the fitted multivariate Gaussian distribution. Here, different subsets are created by (1) the cluster that the current instance belongs to, and (2) the current instance and its k nearest minority neighbors. Furthermore, Euclidean distance-based sample optimization is developed for improved imbalance classification. Finally, late fusion based on majority voting is utilized to obtain final predictions. Experiment results on 15 KEEL datasets demonstrate the great effectiveness of our proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 89.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 119.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Kahl, S., et al.: Large-scale bird sound classification using convolutional neural networks. In: CLEF (Working Notes), vol. 1866 (2017)

    Google Scholar 

  2. Zhu, H., Liu, G., Zhou, M., Xie, Y., Abusorrah, A., Kang, Q.: Optimizing weighted extreme learning machines for imbalanced classification and application to credit card fraud detection. Neurocomputing 407, 50–62 (2020)

    Article  Google Scholar 

  3. Huda, S., et al.: An ensemble oversampling model for class imbalance problem in software defect prediction. IEEE Access 6, 24184–24195 (2018)

    Article  Google Scholar 

  4. Shawky, M.: Factors affecting lane change crashes. IATSS Res. 44, 155–161 (2020)

    Article  Google Scholar 

  5. Han, H., Wang, W.-Y., Mao, B.-H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. In: Huang, D.-S., Zhang, X.-P., Huang, G.-B. (eds.) ICIC 2005. LNCS, vol. 3644, pp. 878–887. Springer, Heidelberg (2005). https://doi.org/10.1007/11538059_91

    Chapter  Google Scholar 

  6. Kang, Q., Chen, X., Li, S., Zhou, M.: A noise-filtered under-sampling scheme for imbalanced classification. IEEE Trans. Cybernet. 47, 4263–4274 (2016)

    Article  Google Scholar 

  7. López, V., Fernández, A., Moreno-Torres, J.G., Herrera, F.: Analysis of preprocessing vs. cost-sensitive learning for imbalanced classification. Open problems on intrinsic data characteristics. Expert Syst. Appl. 39, 6585–6608 (2012)

    Google Scholar 

  8. Liu, Y., Lu, H., Yan, K., Xia, H., An, C.: Applying cost-sensitive extreme learning machine and dissimilarity integration to gene expression data classification. Comput. Intell. Neurosci. 2016 (2016)

    Google Scholar 

  9. Khan, S.H., Hayat, M., Bennamoun, M., Sohel, F.A., Togneri, R.: Cost-sensitive learning of deep feature representations from imbalanced data. IEEE Trans. Neural Netw. Learn. Syst. 29, 3573–3587 (2017)

    Google Scholar 

  10. Li, J., Fong, S., Wong, R.K., Chu, V.W.: Adaptive multi-objective swarm fusion for imbalanced data classification. Inf. Fus. 39, 1–24 (2018)

    Article  Google Scholar 

  11. Chen, R., Guo, S.-K., Wang, X.-Z., Zhang, T.-L.: Fusion of multi-RSMOTE with fuzzy integral to classify bug reports with an imbalanced distribution. IEEE Trans. Fuzzy Syst. 27, 2406–2420 (2019)

    Article  Google Scholar 

  12. Yang, J., Xie, G., Yang, Y.: An improved ensemble fusion autoencoder model for fault diagnosis from imbalanced and incomplete data. Control Eng. Pract. 98, 104358 (2020)

    Article  Google Scholar 

  13. Liu, X.-Y., Wu, J., Zhou, Z.-H.: Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybernet. Part B (Cybernetics) 39, 539–550 (2008)

    Google Scholar 

  14. Seiffert, C., Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: RUSBoost: a hybrid approach to alleviating class imbalance. IEEE Trans. Syst. Man Cybernet. A Syst. Hum. 40, 185–197 (2009)

    Article  Google Scholar 

  15. Wang, S., Yao, X.: Diversity analysis on imbalanced data sets by using ensemble models. In: 2009 IEEE Symposium on Computational Intelligence and Data Mining, pp. 324–331. IEEE (2009)

    Google Scholar 

  16. Chen, C., Liaw, A., Breiman, L.: Using random forest to learn imbalanced data, Technical report (2004)

    Google Scholar 

  17. Chawla, N.V., Lazarevic, A., Hall, L.O., Bowyer, K.W.: SMOTEBoost: improving prediction of the minority class in boosting. In: Lavrač, N., Gamberger, D., Todorovski, L., Blockeel, H. (eds.) PKDD 2003. LNCS (LNAI), vol. 2838, pp. 107–119. Springer, Heidelberg (2003). https://doi.org/10.1007/978-3-540-39804-2_12

    Chapter  Google Scholar 

  18. Maclin, R., Opitz, D.: An empirical evaluation of bagging and boosting. In: AAAI/IAAI 1997, pp. 546–551 (1997)

    Google Scholar 

  19. Khoshgoftaar, T.M., Van Hulse, J., Napolitano, A.: Comparing boosting and bagging techniques with noisy and imbalanced data. IEEE Trans. Syst. Man Cybernet. A Syst. Hum. 41, 552–568 (2010)

    Article  Google Scholar 

  20. Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybernet. C (Appl. Rev.) 42, 463–484 (2011)

    Article  Google Scholar 

  21. Dongdong, L., Ziqiu, C., Bolu, W., Zhe, W., Hai, Y., Wenli, D.: Entropy-based hybrid sampling ensemble learning for imbalanced data. Int. J. Intell. Syst. 36, 3039–3067 (2021)

    Article  Google Scholar 

  22. Lim, P., Goh, C.K., Tan, K.C.: Evolutionary cluster-based synthetic oversampling ensemble (eco-ensemble) for imbalance learning. IEEE Trans. Cybernet. 47, 2850–2861 (2016)

    Article  Google Scholar 

  23. Yan, Y.T., Wu, Z.B., Du, X.Q., Chen, J., Zhao, S., Zhang, Y.P.: A three-way decision ensemble method for imbalanced data oversampling. Int. J. Approx. Reason. 107, 1–16 (2019)

    Article  MathSciNet  Google Scholar 

  24. Gicić, A., Subasi, A.: Credit scoring for a microcredit data set using the synthetic minority oversampling technique and ensemble classifiers. Expert Syst. 36, e12363 (2019)

    Article  Google Scholar 

  25. Zefrehi, H.G., Altınçay, H.: Imbalance learning using heterogeneous ensembles. Expert Syst. Appl. 142, 113005 (2020)

    Google Scholar 

  26. Chen, Z., Duan, J., Kang, L., Qiu, G.: A hybrid data-level ensemble to enable learning from highly imbalanced dataset. Inf. Sci. 554, 157–176 (2021)

    Article  MathSciNet  Google Scholar 

  27. Yuan, B.-W., Zhang, Z.-L., Luo, X.-G., Yu, Y., Zou, X.-H., Zou, X.-D.: OIS-RF: a novel overlap and imbalance sensitive random forest. Eng. Appl. Artif. Intell. 104, 104355 (2021)

    Article  Google Scholar 

  28. Chongomweru, H., Kasem, A.: A novel ensemble method for classification in imbalanced datasets using split balancing technique based on instance hardness (sBAL_IH). Neural Comput. Appl. 33, 1–22 (2021)

    Google Scholar 

  29. Xie, Y., Qiu, M., Zhang, H., Peng, L., Chen, Z.: Gaussian distribution based oversampling for imbalanced data classification. IEEE Trans. Knowl. Data Eng. (2020)

    Google Scholar 

  30. Biernacki, C., Celeux, G., Govaert, G.: Choosing starting values for the EM algorithm for getting the highest likelihood in multivariate gaussian mixture models. Comput. Stat. Data Anal. 41, 561–575 (2003)

    Article  MathSciNet  Google Scholar 

  31. Crump, M.J., Navarro, D., Suzuki, J.: Answering questions with data: introductory statistics for psychology students (2019)

    Google Scholar 

  32. Alcalá-Fdez, J., et al.: Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework. J. Multiple-Valued Logic Soft Comput. 17 (2011)

    Google Scholar 

  33. Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    MathSciNet  MATH  Google Scholar 

  34. Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: Smote: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)

    Article  Google Scholar 

Download references

Acknowledgment

This work is supported by National Natural Science Foundation of China (Grant No: 61902154 and 72004092). This work is also partially supported by Natural Science Foundation of Jiangsu Province (Grant No: BK2019043526), Jiangsu Province Post Doctoral Fund (Grant No: 2020Z430), and China Postdoctoral Science special Foundation (Grant No. 2021T140281).

Author information

Authors and Affiliations

  1. School of Computer and Electronic Information and the School of Artificial Intelligence, Nanjing Normal University, Nanjing, China

    Jie Xie

  2. Key Laboratory of Advanced Process Control for Light Industry (Ministry of Education), School of Internet of Things Engineering, Jiangnan University, Wuxi, 214122, China

    Jie Xie & Kai Hu

  3. School of Economics, Nanjing University, 22 Hankou Road, Nanjing, 210093, Jiangsu, China

    Mingying Zhu

  4. Hopkins-Nanjing Center for Chinese and American Studies, Johns Hopkins University, 162 Shanghai Road Nanjing University, Nanjing, 210093, Jiangsu, China

    Mingying Zhu

Authors
  1. Jie Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mingying Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kai Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mingying Zhu .

Editor information

Editors and Affiliations

  1. Brunel University London, London, UK

    Derek Groen

  2. University of Amsterdam, Amsterdam, The Netherlands

    Clélia de Mulatier

  3. AGH University of Science and Technology, Krakow, Poland

    Maciej Paszynski

  4. University of Amsterdam, Amsterdam, The Netherlands

    Valeria V. Krzhizhanovskaya

  5. University of Tennessee at Knoxville, Knoxville, TN, USA

    Jack J. Dongarra

  6. University of Amsterdam, Amsterdam, The Netherlands

    Peter M. A. Sloot

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xie, J., Zhu, M., Hu, K. (2022). Hierarchical Ensemble Based Imbalance Classification. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13350. Springer, Cham. https://doi.org/10.1007/978-3-031-08751-6_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-08751-6_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-08750-9

  • Online ISBN: 978-3-031-08751-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation