Skip to main content
SpringerLink
Log in

Bee: towards a robust attribute reduction

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The problem solving of attribute reduction is popular in reducing dimensions of data. Note that besides the efficiency of searching expected reducts, the performance related to the derived reducts should also be paid much attention to. Among various representative performance, it is worth mentioning that the robustness of reduct is crucial to downstream learning tasks. The reason is contributed to the fact that unstable results of attribute reduction may shake the confidence of domain experts when experimentally validating the selected attributes in reducts. In view of this, a novel framework called Bucket based Ensemble sElector (Bee) was developed, which outputs robust reduct with higher stability. Firstly, raw sample space was partitioned by a bucket mechanism. Secondly, over each bucket, candidate attributes were evaluated and then an appropriate attribute was identified. Finally, a voting was executed to identify a universal attribute which should be added into reduct pool for each iteration in the process of searching. Additionally, our framework was introduced into not only the searchings of approximation quality, regularization loss, unsupervised relevance related reducts, but also a quick searching procedure called attribute group. By testing 20 UCI benckmark data sets with raw label and 4 different ratios (10%, 20%, 30%, 40%) of noisy label, comprehensive experiments demonstrated the superiorities of our Bee: it not only offers robust results of attribute reduction but also guarantees comparable predictions than some other popular algorithms.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. Ba J, Liu KY, Ju HR, Xu SP, Xu TH, Yang XB (2022) Triple-G: a new MGRS and attribute reduction. Int J Mach Learn Cybern 13:337–356

    Article  Google Scholar 

  2. Bania RK, Hahn RK, Halder A (2020) R-Ensembler: a greedy rough set based ensemble attribute selection algorithm with KNN imputation for classification of medical data. Comput Methods Prog Biomed 184: 105122

  3. Binato S, de Oliveira GC, de Araujo JL (2001) A greedy randomized adaptive search procedure for transmission expansion planning. IEEE Trans Power Syst 16:247–253

    Article  Google Scholar 

  4. Breiman LI, Friedman JH, Olshen RA, Stone CJ (1984) Classification and regression trees. Biometrics 40:358

    MathSciNet  MATH  Google Scholar 

  5. Cai J, Luo J, Wang S, Yang S (2018) Feature selection in machine learning: a new perspective. Neurocomputing 300:70–79

    Article  Google Scholar 

  6. Chen DG, Yang YY (2014) Attribute reduction for heterogeneous data on the combination of classical and fuzzy rough set models. IEEE Trans Fuzzy Syst 22:1325–1334

    Article  Google Scholar 

  7. Chen DG, Zhao SY, Zhang L, Yang YP, Zhao X (2012) Sample pair selection for attribute reduction with rough set. IEEE Trans Knowl Data Eng 24:2080–2093

    Article  Google Scholar 

  8. Chen DG, Yang YY, Dong Z (2016) An incremental algorithm for attribute reduction with variable precision rough sets. Appl Soft Comput 45:129–149

    Article  Google Scholar 

  9. Chen Y, Liu KY, Song JJ, Fujita H, Yang XB, Qian YH (2020) Attribute group for attribute reduction. Inform Sci 535:64–80

    Article  MATH  Google Scholar 

  10. Chen Y, Wang PX, Yang XB, Mi JS, Liu D (2021) Granular ball guided selector for attribute reduction. Knowledge-Based Systems 229: 107326

  11. Chen Y, Yang XB, Li JH, Wang PX, Qian YH (2022) Fusing attribute reduction accelerators. Inform Sci 587:354–370

    Article  Google Scholar 

  12. Chen Z, Liu KY, Yang XB, Fujita H (2022) Random sampling accelerator for attribute reduction. Int J Approx Reason 140:75–91

    Article  MathSciNet  MATH  Google Scholar 

  13. Cover TM, Hart PE (1967) Nearest neighbor pattern classification. IEEE Trans Inform Theory 13:21–27

    Article  MATH  Google Scholar 

  14. Dong LJ, Chen DG (2020) Incremental attribute reduction with rough set for dynamic datasets with simultaneously increasing samples and attributes. Int J Mach Learn Cybern 11:1339–1355

    Article  Google Scholar 

  15. Dong LJ, Chen DG, Wang NL, Lu ZH (2020) Key energy consumption feature selection of thermal power systems based on robust attribute reduction with rough sets. Inform Sci 532:61–71

    Article  Google Scholar 

  16. Dong F, Lu J, Song Y, Liu F, Zhang G (2021) A drift region based data sample filtering method. IEEE Trans Cybern 99:1–14

    Article  Google Scholar 

  17. Hu QH, Pedrycz W, Yu DR, Lang J (2009) Selecting discrete and continuous features based on neighborhood decision error minimization. IEEE Trans Syst Man Cybern Part B (Cybern) 40:137–150

    Google Scholar 

  18. Hu QH, An S, Yu X, Yu DR (2011) Robust fuzzy rough classifiers. Fuzzy Sets Syst 183:26–43

    Article  MathSciNet  MATH  Google Scholar 

  19. Huang QQ, Li TR, Huang YY, Yang X, Fujita H (2020) Dynamic dominance rough set approach for processing composite ordered data. Knowl Based Syst 187: 104829

  20. Jia XY, Rao Y, Shang L, Li TJ (2020) Similarity based attribute reduction in rough set theory: a clustering perspective. Int J Mach Learn Cybern 11:1047–1060

    Article  Google Scholar 

  21. Jiang GX, Wang WJ (2017) Markov cross validation for time series model evaluations. Inform Sci 375:219–233

    Article  MATH  Google Scholar 

  22. Jiang ZH, Yang XB, Yu HL, Liu D, Wang PX, Qian YH (2019) Accelerator for multi-granularity attribute reduction. Knowl Based Syst 177:145–158

    Article  Google Scholar 

  23. Jiang ZH, Liu KY, Yang XB, Yu HL, Fujita H, Qian YH (2020) Accelerator for supervised neighborhood based attribute reduction. Int J Approx Reason 119:122–150

    Article  MathSciNet  MATH  Google Scholar 

  24. Jiang ZH, Dou HL, Song JJ, Wang PX, Yang XB, Qian YH (2021) Data-guided multi-granularity selector for attribute reduction. Appl Intell 51:876–888

    Article  Google Scholar 

  25. Jiang ZH, Liu KY, Song JJ, Yang XB, Li JH, Qian YH (2021) Accelerator for crosswise computing reduct. Appl Soft Comput 98: 106740

  26. Jin CX, Li FC, Hu QH (2017) Knowledge change rate-based attribute importance measure and its performance analysis. Knowl Based Syst 119:59–67

    Article  Google Scholar 

  27. Ju HR, Yang XB, Song XN, Qi YS (2014) Dynamic updating multigranulation fuzzy rough set: approximations and reducts. Int J Mach Learn Cybern 5:981–990

    Article  Google Scholar 

  28. Kalousis A, Prados J, Hilario M (2005) Stability of feature selection algorithms. In: 5th IEEE International Conference on Data Mining, pp 8–16

  29. Li Y, Si J, Zhou GJ, Huang SS, Chen SC (2014) FREL: a stable feature selection algorithm. IEEE Trans Neural Netw Learn Syst 26:1388–1402

    Article  MathSciNet  Google Scholar 

  30. Li JZ, Yang XB, Song XN, Li JH, Wang PX, Yu DJ (2019) Neighborhood attribute reduction: a multi-criterion approach. Int J Mach Learn Cybern 10:731–742

    Article  Google Scholar 

  31. Liang JY, Chin KS, Dang CY, Yam Richid CM (2002) A new method for measuring uncertainty and fuzziness in rough set theory. Int J Gen Syst 31:331–342

    Article  MathSciNet  MATH  Google Scholar 

  32. Liu Y, Huang WL, Jiang YL, Zeng ZY (2014) Quick attribute reduct algorithm for neighborhood rough set model. Inform Sci 271:65–81

    Article  MathSciNet  MATH  Google Scholar 

  33. Liu KY, Yang XB, Fujita H, Liu D, Yang X, Qian YH (2019) An efficient selector for multi-granularity attribute reduction. Inform Sci 505:457–472

    Article  Google Scholar 

  34. Liu KY, Yang XB, Yu HL, Mi JS, Wang PX, Chen XJ (2019) Rough set based semi-supervised feature selection via ensemble selector. Knowl Based Syst 165:282–296

    Article  Google Scholar 

  35. Liu KY, Yang XB, Yu HL, Fujita H, Chen XJ, Liu D (2020) Supervised information granulation strategy for attribute reduction. Int J Mach Learn Cybern 11:2149–2163

    Article  Google Scholar 

  36. Liu YX, Gong ZC, Liu KY, Xu SP, Ju HR, Yang XB (2022) A Q-learning approach to attribute reduction. Appl Intell. https://doi.org/10.1007/s10489-022-03696-w

    Article  Google Scholar 

  37. Liu KY, Li TR, Yang XB, Yang X, Liu D, Zhang PF, Wang J (2022) Granular cabin: an efficient solution to neighborhood learning in big data. Inform Sci 583:189–201

    Article  Google Scholar 

  38. Mi YL, Shi Y, Li JH, Liu WQ, Yan MY (2022) Fuzzy-based concept learning method: Exploiting data with fuzzy conceptual clustering. IEEE Trans Cybern 52:582–593

    Article  Google Scholar 

  39. Min F, He HP, Qian YH, Zhu W (2011) Test-cost-sensitive attribute reduction. Inform Sci 181:4928–4942

    Article  Google Scholar 

  40. Naik AK, Kuppili V, Edla DR (2020) A new hybrid stability measure for feature selection. Appl Intell 50:3471–3486

    Article  Google Scholar 

  41. Nogueira S, Sechidis K, Brown G (2018) On the stability of feature selection algorithms. J Mach Learn Res 18:1–54

    MathSciNet  MATH  Google Scholar 

  42. Paulus J, Klapuri A (2009) Music structure analysis using a probabilistic fitness measure and a greedy search algorithm. IEEE Trans Audio Speech Lang Process 17:1159–1170

    Article  Google Scholar 

  43. Pawlak Z (1982) Rough sets. Int J Comput Inform Sci 11:342–356

    Article  MATH  Google Scholar 

  44. Qian YH, Liang JY, Pedrycz W, Dang CY (2010) Positive approximation: an accelerator for attribute reduction in rough set theory. Artif Intell 174:597–618

    Article  MathSciNet  MATH  Google Scholar 

  45. Qian YH, Wang Q, Cheng HH, Liang JY, Dang CY (2015) Fuzzy-rough feature selection accelerator. Fuzzy Sets Syst 258:61–78

    Article  MathSciNet  MATH  Google Scholar 

  46. Qu YP, Xu Z, Shang CJ, Ge XL, Deng AS, Shen Q (2021) Inconsistency guided robust attribute reduction. Inform Sci 580:69–91

    Article  MathSciNet  Google Scholar 

  47. Rao XS, Yang XB, Yang X, Chen XJ, Liu D, Qian YH (2020) Quickly calculating reduct: an attribute relationship based approach. Knowl Based Syst 200: 106014

  48. Sarkar C, Cooley S, Srivastava J (2014) Robust feature selection technique usingrank aggregation. Appl Artif Intell 28:243–257

    Article  Google Scholar 

  49. Tsang ECC, Hu QH, Chen DG (2016) Feature and instance reduction for PNNclassifiers based on fuzzy rough sets. Int J Mach Learn Cybern 7:1–11

    Article  Google Scholar 

  50. Wang PX, Wu TF (2021) Three-way clustering method based on stability theory. IEEE Access 9:33944–33953

    Article  Google Scholar 

  51. Wang PX, Yao YY (2018) CE3: a three-way clustering method based on mathematical morphology. Knowl Based Syst 155:54–65

    Article  Google Scholar 

  52. Wang CZ, Shi YP, Fan XD, Shao MW (2018) Attribute reduction based on $k$-nearest neighborhood rough sets. Int J Approx Reason 106:18–31

    Article  MathSciNet  MATH  Google Scholar 

  53. Wang YB, Chen XJ, Dong K (2019) Attribute reduction via local conditional entropy. Int J Mach Learn Cybern 10:3619–3634

    Article  Google Scholar 

  54. Wang PX, Shi H, Yang XB, Mi JS (2019) Three-way $k$-means: integrating $k$-means and three-way decision. Int J Mach Learn Cybern 10:2767–2777

    Article  Google Scholar 

  55. Wang X, Wang PX, Yang XB, Yao YY (2021) Attribution reduction based on sequential three-way search of granularity. Int J Mach Learn Cybern 12:1439–1458

    Article  Google Scholar 

  56. Wei W, Liang JY (2019) Information fusion in rough set theory: an overview. Inform Fus 48:107–118

    Article  Google Scholar 

  57. Xia SY, Liu YS, Ding X, Wang GY, Yu H, Luo YG (2019) Granular ball computing classifiers for efficient, scalable and robust learning. Inform Sci 483:136–152

    Article  MathSciNet  Google Scholar 

  58. Xia SY, Dai XC, Wang GY, Gao XB, Giem E (2022) An efficient and adaptive granular ball generation method in classification problem. arXiv preprint arXiv:2201.04343

  59. Xia SY, Wang C, Wang GY, Ding WP, Gao XB, Yu JH, Zhai YJ, Chen ZZ (2022) A unified granular-ball learning model of pawlak rough set and neighborhood rough set. arXiv preprint arXiv:2201.03349

  60. Xia SY, Zhang Z, Li WH, Wang GY, Giem E, Chen ZZ (2022) GBNRS: a novel rough set algorithm for fast adaptive attribute reduction in classification. IEEE Trans Knowl Data Eng 34:1231–1242

    Article  Google Scholar 

  61. Xu SP, Yang XB, Yu HL, Yu DJ, Yang JY, Tsang ECC (2016) Multi-label learning with label-specific feature reduction. Knowl Based Syst 104:52–61

    Article  Google Scholar 

  62. Xu SP, Ju HR, Shang L, Pedrycz W, Yang XB, Li C (2020) Label distribution learning: a local collaborative mechanism. Int J Approx Reason 121:59–84

    Article  MathSciNet  Google Scholar 

  63. Yang XB, Yao YY (2018) Ensemble selector for attribute reduction. Appl Soft Comput 70:1–11

    Article  Google Scholar 

  64. Yang XB, Qi YS, Song XN, Yang JY (2013) Test cost sensitive multigranulation rough set: model and minimal cost selection. Inform Sci 250:184–199

    Article  MathSciNet  MATH  Google Scholar 

  65. Yang XB, Liang SC, Yu HL, Gao S, Qian YH (2019) Pseudo-label neighborhood rough set: measures and attribute reductions. Int J Approx Reason 105:112–129

    Article  MathSciNet  MATH  Google Scholar 

  66. Yang X, Liu D, Yang XB, Liu KY, Li TR (2021) Incremental fuzzy probability decision-theoretic approaches to dynamic three-way approximations. Inform Sci 550:71–90

    Article  MathSciNet  MATH  Google Scholar 

  67. Yang X, Li MM, Fujita H, Liu D, Li TR (2022) Incremental rough reduction with stable attribute group. Inform Sci 589:283–299

    Article  Google Scholar 

  68. Yang X, Yang YX, Luo JF, Liu D, Li TR (2022) A unified incremental updating framework of attribute reduction for two-dimensionally time-evolving data. Inform Sci 601:287–305

    Article  Google Scholar 

  69. Yao YY, Zhang XY (2017) Class-specfic attribute reducts in rough set theory. Inform Sci 418:601–618

    Article  MATH  Google Scholar 

  70. Yuan Z, Chen HM, Tian TR, Yu Z, Sang BB, Luo C (2021) Unsupervised attribute reduction for mixed data based on fuzzy rough sets. Inform Sci 572:67–87

    Article  MathSciNet  Google Scholar 

  71. Zhang YY, Liu F, Fang Z, Yuan B, Zhang GQ, Lu J (2021) Learning from a complementary-label source domain: theory and algorithms. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2021.3086093

    Article  Google Scholar 

  72. Zhu PF, Xu Q, Hu QH, Zhang CQ (2018) Co-regularized unsupervised feature selection. Neurocomputing 275:2855–2863

    Article  Google Scholar 

  73. Zhu PF, Li JL, Wang Y, Xiao B, Zhao S, Hu QH (2022) Collaborative decision-reinforced self-supervision for attributed graph clustering. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/tnnls.2022.3171583

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the editors and anonymous reviewers for their constructive comments. This work was supported by the Natural Science Foundation of China (Nos. 62076111, 62176107, 62006099, 61906078) and the Postgraduate Research & Practice Innovation Program of Jiangsu Province (No. SJX22_1900).

Author information

Authors and Affiliations

  1. School of Computer, Jiangsu University of Science and Technology, Zhenjiang, 212100, Jiangsu, China

    Yining Chen, Xibei Yang & Hualong Yu

  2. School of Sciences, Jiangsu University of Science and Technology, Zhenjiang, 212100, Jiangsu, China

    Pingxin Wang & Xibei Yang

  3. Key Laboratory of Oceanographic Big Data Mining and Application of Zhejiang Province, Zhoushan, 316022, Zhejiang, China

    Pingxin Wang

Authors
  1. Yining Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pingxin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xibei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hualong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xibei Yang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chen, Y., Wang, P., Yang, X. et al. Bee: towards a robust attribute reduction. Int. J. Mach. Learn. & Cyber. 13, 3927–3962 (2022). https://doi.org/10.1007/s13042-022-01633-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-022-01633-4

Keywords

Search

Navigation