Skip to main content
SpringerLink
Log in

Three-way selection random forest algorithm based on decision boundary entropy

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aiming at the problem of high probability of negative impact about redundant attributes in random forest algorithms, a Three-way Selection Random Forest algorithm based on decision boundary entropy (TSRF) is proposed without losing randomness and reducing the influence of redundant attributes on decision-making results. According to the characteristics of the attribute, the concept of decision boundary entropy is defined. Then a measuring method of attribute importance based on decision boundary entropy is proposed and set as an evaluation standard. Three-way decision is constructed and the attribute is divided into three candidate domains, namely positive domain, negative domain and boundary domain. In order to ensure the randomness of attributes, three-way attribute random selection rules based on attribute randomness are established and a certain number of attributes are randomly selected from the three candidate domains. Combine the samples selected by the bootstrap sampling method with attribute sets selected by three-way decision to produce training sample sets so that we can train the decision trees and generate forest. Six datasets are selected for the experiment. Two parameters of attribute randomness and three-way decision thresholds are analyzed to verify the theoretical conclusions respectively. The results show that the TSRF algorithm can meet the different requirements of different data sets by adjusting the parameters. The classification effect on the binary data is basically the same as the comparison algorithm, but TSRF has a significant improvement effect on the multi-class data compared with other algorithms. The proposed TSRF algorithm widens the idea for the measurement method of significance of attribute, innovates the random forest three-way selection integration method, and provides a better model framework for solving multi-classification problems.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Yang AM, Zhuansun YX, Shi Y, Liu HX, Li RS (2021) Iot System For Pellet Proportioning Based On Bas Intelligent Recommendation Model. IEEE Trans Ind Inform 17(2):937–942

    Article  Google Scholar 

  2. Pan YH, Wang YH, Zhou P, Yan Y, Guo DM (2020) Activation Functions Selection For BP Neural Network Model Of Ground Surface Roughness. J Intell Manuf 31(8):1825–1836

    Article  Google Scholar 

  3. Yan SH, Yang AM, Kong SH, Bai B, Li Xi Y (2021) Predictive Intelligence Powered Attentional Stacking Matrix Factorization Algorithm For The Computational Drug Repositioning. Appl Soft Comput J 110:107633

    Article  Google Scholar 

  4. Leandro VK, Wang Y, Gabriela H (2021) Online Ensemble Learning For Load Forecasting. IEEE Trans Power Syst 36(1):545–548

    Article  Google Scholar 

  5. Lukasz G, Zenon A. S (2021) Pruning Trees In C-Fuzzy Random Forest. Soft Comput 25(3): 1995-2013.

  6. Pablo D, Juan C. S, Aldo C, Felipe N (2021) Random Forest Model Predictive Control For Paste Thickening. Miner Eng 163: 106760.

  7. Shitanshu D, Taha BMJO (2021) Regional Hydrological Frequency Analysis At Ungauged Sites With Random Forest Regression. J Hydrol 594:125861

    Article  Google Scholar 

  8. Hill AJ, Heman GR, Schumacher RS (2020) Forecasting Severe Weather With Random Forests. Mon Weather Rev 148(5):2135–2161

    Article  Google Scholar 

  9. Martin K, Tomáš P, Martin H (2020) Anomaly Explanation With Random Forests. Expert Syst Appl 149:113187

    Article  Google Scholar 

  10. Hadi TM, Mohammad D (2018) Combining Emerging Patterns With Random Forest For Complex Activity Recognition In Smart Homes. Appl Intell 48(2):315–330

    Article  Google Scholar 

  11. Xia SY, Wang GY, Chen ZZ, Duan YL, Liu Q (2018) Complete Random Forest Based Class Noise Filtering Learning For Improving The Generalizability Of Classifiers. IEEE Trans Knowl Data Eng 31(11):2063–2078

    Article  Google Scholar 

  12. Xu H, Wang WJ, Qian YH (2017) Fusing Complete Monotonic Decision Trees. IEEE Trans Knowl Data Eng 29(10):2223–2235

    Article  Google Scholar 

  13. Zhong Y, Yang HY, Zhang YC, Li P (2021) Online Rebuilding Regression Random Forests. Knowl-Based Syst 221:106960

    Article  Google Scholar 

  14. Ao YL, Li HQ, Zhu LP, Sikandar A, Yang ZG (2018) Identifying Channel Sand-Body From Multiple Seismic Attributes With An Improved Random Forest Algorithm. J Pet Sci Eng 173:781–792

    Article  Google Scholar 

  15. Liang BH, Wang L, Liu Y (2019) Attribute Reduction Based On Improved Information Entropy. J Intell Fuzzy Syst 36(1):709–718

    Article  Google Scholar 

  16. Li FC, Jin CX, Zhang X, Wang Y, Liu XF (2020) Attribute Importance Measurement Method Based On Data Coordination Degree. Knowl-Based Syst 192:105359

    Article  Google Scholar 

  17. Liu JF, Bai ML, Jiang N, Yu D (2019) A Novel Measure Of Attribute Significance With Complexity Weight. Appl Soft Comput 82:105543

    Article  Google Scholar 

  18. Ayşegül AU, Murat D (2020) Generalized Textural Rough Sets: Rough Set Models Over Two Universes. Inf Sci 521:398–421

    Article  MathSciNet  MATH  Google Scholar 

  19. Zhang PF, Li TR, Wang GQ, Luo C, Chen HM, Zhang JB, Wang DX, Yu Z (2021) Multi-Source Information Fusion Based On Rough Set Theory: A Review. Inf Fusion 68:85–117

    Article  Google Scholar 

  20. An S, Hu QH, Wang CZ (2021) Probability Granular Distance-Based Fuzzy Rough Set Model. Appl Soft Comput J 102:107064

    Article  Google Scholar 

  21. Sang-Eon H (2021) Topological Properties Of Locally Finite Covering Rough Sets And K-Topological Rough Set Structures. Soft Comput 25(10):6865–6877

    Article  Google Scholar 

  22. Chakhar S, Ishizaka A, Thorpe A, Cox J, Nguyen T, Ford L (2020) Calculating The Relative Importance Of Condition Attributes Based On The Characteristics Of Decision Rules And Attribute Reducts: Application To Crowdfunding. Eur J Oper Res 286(2):689–712

    Article  MathSciNet  MATH  Google Scholar 

  23. Hu M, Tsang E C.C., Guo Y T, Chen D G, Xu W H (2021) A Novel Approach To Attribute Reduction Based On Weighted Neighborhood Rough Sets. Knowl-Based Syst 220: 106908.

  24. Wang JT, Qian YH, Li FJ, Liang JY, Ding WP (2020) Fusing Fuzzy Monotonic Decision Trees. IEEE Trans Fuzzy Syst 28(5):887–900

    Article  Google Scholar 

  25. Li F, Hu BQ, Wang J (2017) Stepwise Optimal Scale Selection For Multi-Scale Decision Tables Via Attribute Significance. Knowl-Based Syst 129:4–16

    Article  Google Scholar 

  26. Li XN, Wang X, Sun BZ, She YH, Zhao L (2021) Three-Way Decision On Information Tables. Inf Sci 545:25–43

    Article  MathSciNet  MATH  Google Scholar 

  27. Qiao J, Hu BQ (2018) On Transformations From Semi-Three-Way Decision Space To Three-Way Decision Spaces Based On Triangular Norms And Triangular Conforms. Inf Sci 432:22–51

    Article  MATH  Google Scholar 

  28. Hu BQ (2017) Three-Way Decision Based On Semi-Three-Way Decision Space. Inf Sci 382-383:415–440

    Article  MATH  Google Scholar 

  29. Zhang YJ, Miao DQ, Zhang ZF, Xu JF, Luo S (2018) A Three-Way Selective Ensemble Model For Multi-Label Classification. Int J Approx Reason 103:394–413

    Article  MathSciNet  MATH  Google Scholar 

  30. Yang B, Li JH (2020) Complex Network Analysis Of Three-Way Decision Researches. Int J Mach Learn Cybern 11(5):973–987

    Article  Google Scholar 

  31. Li ZW, Xie NX, Huang D, Zhang GQ (2020) A Three-Way Decision Method In A Hybrid Decision Information System And Its Application In Medical Diagnosis. Artif Intell Rev 53(7):1–30

    Article  Google Scholar 

  32. Li ZW, Zhang PF, Xie NX, Zhang GQ, Wen CF (2020) A Novel Three-Way Decision Method In A Hybrid Information System With Images And Its Application In Medical Diagnosis. Eng Appl Artif Intell 92:103651

    Article  Google Scholar 

  33. Li X, Wang H, Xu ZS (2021) Work Resumption After Epidemic Using Three-Way Decisions. Int J Fuzzy Syst 23(3):630–641

    Article  Google Scholar 

  34. Li YF, Zhang LB, Xu Y, Yao YY (2017) Enhancing Binary Classification By Modeling Uncertain Boundary In Three-Way Decisions. IEEE Trans Knowl Data Eng 29(7):1438–1451

    Article  Google Scholar 

  35. Zhang B, Wei ZY, Ren JD, Yq C, Zheng ZQ (2018) An Empirical Study On Predicting Blood Pressure Using Classification And Regression Trees. IEEE Access 6:1

    Article  Google Scholar 

  36. Jia K, Zhu ZX, Yang Z, Fang Y, Bi TS, Zhang JK (2020) Intelligent Islanding Detection Method For Photovoltaic Power System Based On Adaboost Algorithm. IET Gener Transm Distrib 14(18):3630–3640

    Article  Google Scholar 

  37. Zhang XH, Wu FM, Li ZR (2021) Application Of Convolutional Neural Network To Traditional Data. Expert Syst Appl 168:114185

    Article  Google Scholar 

  38. Qiao JS, Hu BQ (2020) On Decision Evaluation Functions In Generalized Three-Way Decision Spaces. Inf Sci 507:733–754

    Article  MathSciNet  MATH  Google Scholar 

  39. Jia ZH, Qiao JS (2020) On Decision Evaluation Functions In Three-Way Decision Spaces Derived From Overlap And Grouping Functions. Soft Comput 24(20):15159–15178

    Article  MATH  Google Scholar 

  40. Jiang F, Sui YF, Zhou L (2015) A Relative Decision Entropy-Based Feature Selection Approach. Pattern Recogn 48:2151–2163

    Article  MATH  Google Scholar 

Download references

Acknowledgments

This paper was supported by the Natural Science Foundation of Hebei Province under Grant No. F2018209374.

Author information

Authors and Affiliations

  1. College of Science, North China University of Science and Technology, Tangshan, 063210, Hebei, China

    Chunying Zhang, Jing Ren, Xiaoqi Li & Shouyue Liu

  2. Key Laboratory of Data Science and Application of Hebei Province, Tangshan, 063210, Hebei, China

    Chunying Zhang

  3. College of Qian’an, North China University of Science and Technology, Tangshan, 063210, Hebei, China

    Fengchun Liu

Authors
  1. Chunying Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jing Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fengchun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoqi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Shouyue Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Fengchun Liu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the Topical Collection on Big Data-Driven Large-Scale Group Decision Making Under Uncertainty

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, C., Ren, J., Liu, F. et al. Three-way selection random forest algorithm based on decision boundary entropy. Appl Intell 52, 13384–13397 (2022). https://doi.org/10.1007/s10489-021-03033-7

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03033-7

Keywords

Search

Navigation