Abstract
The fuzzy neighborhood rough set integrates the strengths of fuzzy rough set and neighborhood rough set, serving as a pivotal extension of the rough set theory in attribute reduction. However, this model’s widespread application is hindered by its sensitivity to data distribution and limited efficacy in assessing classification uncertainty for datasets with substantial density variations. To mitigate these challenges, this paper introduces an attribute reduction algorithm based on fuzzy neighborhood relative decision mutual information. Firstly, the classification uncertainty of samples is initially defined in terms of relative distance. Simultaneously, the similarity relationship of fuzzy neighborhoods is reformulated, thereby reducing the risk of sample misclassification through integration with variable-precision fuzzy neighborhood rough approximation. Secondly, the notion of representative sample is introduced, leading to a redefinition of fuzzy membership. Thirdly, fuzzy neighborhood relative mutual information from the information view is constructed and combined with fuzzy neighborhood relative dependency from the algebraic view to propose fuzzy neighborhood relative decision mutual information. Finally, an attribute reduction algorithm is devised based on fuzzy neighborhood relative decision mutual information. This algorithm evaluates the significance of attributes by integrating both informational and algebraic perspectives. Comparative tests on 12 public datasets are conducted to assess existing attribute approximation algorithms. The experimental results show that the proposed algorithm achieved an average classification accuracy of 91.28\(\%\) with the KNN classifier and 89.86\(\%\) with the CART classifier. In both classifiers, the algorithm produced an average reduced subset size of 8.54. While significantly reducing feature redundancy, the algorithm consistently maintains a high level of classification accuracy.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The low-dimensional datasets from the experiment were downloaded from http://archive.ics.uci.edu/ml/index.php and the high-dimensional datasets were downloaded from http://csse.szu.edu.cn/staff/zhuzx/.
References
Xing Y, Kochunov P, Van T et al (2022) A novel neighborhood rough set-based feature selection method and its application to biomarker identification of schizophrenia. IEEE J Biomed Health Inform 27(1):215–226. https://doi.org/10.1109/JBHI.2022.3212479
Wang C, Hu Q, Wang X et al (2017) Feature selection based on neighborhood discrimination index. IEEE Trans Neural Netw Learn Syst 29(7):2986–2999. https://doi.org/10.1109/TNNLS.2017.2710422
Yin T, Chen H, Yuan Z et al (2023) Noise-resistant multilabel fuzzy neighborhood rough sets for feature subset selection. Inf Sci 621:200–226. https://doi.org/10.1016/j.ins.2022.11.060
Wang Y, Sun M, Long L et al (2023) Feature gene selection based on fuzzy neighborhood joint entropy. Complex Intell Syst 1–16. https://doi.org/10.1007/s40747-023-01138-9
Wang C, Qian Y, Ding W et al (2021) Feature selection with fuzzy-rough minimum classification error criterion. IEEE Trans Fuzzy Syst 30(8):2930–2942. https://doi.org/10.1109/TFUZZ.2021.3097811
Zhang P, Li T, Wang G et al (2021) Multi-source information fusion based on rough set theory: A review. Inf Fusion 68(1):85–117. https://doi.org/10.1016/j.inffus.2020.11.004
Wang Z, Chen H, Yuan Z et al (2023) Fuzzy-rough hybrid dimensionality reduction. Fuzzy Sets Syst 459:95–117. https://doi.org/10.1016/j.fss.2022.08.004
Gao W, Hu L, Zhang P et al (2018) Feature selection considering the composition of feature relevancy. Pattern Recogn Lett 112:70–74. https://doi.org/10.1016/j.patrec.2018.06.005
Luo C, Wang S, Li T et al (2023) Rhdofs: A distributed online algorithm towards scalable streaming feature selection. IEEE Trans Parallel Distrib Syst. https://doi.org/10.1109/TPDS.2023.3265974
Liang S, Liu Z, You D et al (2023) Pso-nrs: an online group feature selection algorithm based on pso multi-objective optimization. Appl Intell 53(12):15095–15111. https://doi.org/10.1007/s10489-022-04275-9
Zadeh LA (1965) Fuzzy sets. Inf Control 8(3):338–353. https://doi.org/10.1016/S0019-9958(65)90241-X
Wang C, Qi Y, Shao M et al (2016) A fitting model for feature selection with fuzzy rough sets. IEEE Trans Fuzzy Syst 25(4):741–753. https://doi.org/10.1109/TFUZZ.2016.2574918
Wang Z, Chen H, Yuan Z et al (2022) Exploiting fuzzy rough mutual information for feature selection. Appl Soft Comput 131:109769. https://doi.org/10.1016/j.asoc.2022.109769
Pawlak Z (1982) Rough sets. Int J Comput Inf Sci 11:341–356. https://doi.org/10.1007/BF01001956
Chen Y, Chen Y (2021) Feature subset selection based on variable precision neighborhood rough sets. Int J Comput Intell Syst 14(1):572–581. https://doi.org/10.2991/ijcis.d.210106.003
Xia S, Wu S, Chen X et al (2022) Grrs: Accurate and efficient neighborhood rough set for feature selection. IEEE Trans Knowl Data Eng 35(9):9281–9294. https://doi.org/10.1109/TKDE.2022.3222447
Zhang P, Li T, Yuan Z et al (2022) Heterogeneous feature selection based on neighborhood combination entropy. IEEE Trans Neural Netw Learn Syst 1–14. https://doi.org/10.1109/TNNLS.2022.3193929
Hu Q, Pedrycz W, Yu D et al (2009) Selecting discrete and continuous features based on neighborhood decision error minimization. IEEE Trans Syst Man Cybern B Cybern 40(1):137–150. https://doi.org/10.1109/TSMCB.2009.2024166
Wang X, Tsang E, Zhao S et al (2007) Learning fuzzy rules from fuzzy samples based on rough set technique. Inf Sci 177(20):4493–4514. https://doi.org/10.1016/j.ins.2007.04.010
Chen Y, Xue Y, Ma Y et al (2008) Measures of uncertainty for neighborhood rough sets. Knowl-Based Syst 178(18):3577–3594. https://doi.org/10.1016/j.ins.2008.05.024
Xu F, Cai M, Li Q et al (2023) Shared neighbors rough set model and neighborhood classifiers. Expert Syst Appl 244:122965. https://doi.org/10.1016/j.eswa.2023.122965
Sun L, Wang L, Ding W et al (2020) Neighborhood multi-granulation rough sets-based attribute reduction using lebesgue and entropy measures in incomplete neighborhood decision systems. Knowl-Based Syst 192:105373. https://doi.org/10.1016/j.knosys.2019.105373
Zhang X, Gou H, Lv Z et al (2021) Double-quantitative distance measurement and classification learning based on the tri-level granular structure of neighborhood system. Knowl-Based Syst 217:106799. https://doi.org/10.1016/j.knosys.2021.106799
Xu W, Yuan K, Li W et al (2023) An emerging fuzzy feature selection method using composite entropy-based uncertainty measure and data distribution. IEEE Trans Emerg Top Comput Intell 7(1):76–88. https://doi.org/10.1109/TETCI.2022.3171784
Hu Q, Yu D, Liu J et al (2008) Neighborhood rough set based heterogeneous feature subset selection. Inf Sci 178(18):3577–3594. https://doi.org/10.1016/j.ins.2008.05.024
Thuy N, Sartra W (2021) A novel feature selection method for high-dimensional mixed decision tables. IEEE Trans Neural Netw Learn Syst 33(7):3024–3037. https://doi.org/10.1109/TNNLS.2020.3048080
Yang X, Li T, Liu D et al (2020) A multilevel neighborhood sequential decision approach of three-way granular computing. Inf Sci 538:119–141. https://doi.org/10.1016/j.ins.2020.05.060
Chen Y, Liu K, Song J et al (2020) Attribute group for attribute reduction. Inf Sci 535:64–80. https://doi.org/10.1016/j.ins.2020.05.010
Dubois D, Prade H (1990) Rough fuzzy sets and fuzzy rough sets. Int J Gen Syst 17(2–3):191–209. https://doi.org/10.1080/03081079008935107
Yao Y (2010) Three-way decisions with probabilistic rough sets. Inf Sci 180(3):341–353. https://doi.org/10.1016/j.ins.2009.09.021
Hu Q, Zhang L, Chen D et al (2010) Gaussian kernel based fuzzy rough sets: model, uncertainty measures and applications. Int J Approx Reason 51(4):453–471
Lin G, Qian Y, Li J (2012) Nmgrs: Neighborhood-based multigranulation rough sets. Int J Approx Reason 53(7):1080–1093. https://doi.org/10.1016/j.ijar.2012.05.004
Wang C, Shao M, He Q et al (2016) Feature subset selection based on fuzzy neighborhood rough sets. Knowl-Based Syst 111:173–179. https://doi.org/10.1016/j.knosys.2016.08.009
Zhang X, Li J (2023) Incremental feature selection approach to interval-valued fuzzy decision information systems based on fuzzy similarity self-information. Inf Sci 625:593–619. https://doi.org/10.1016/j.ins.2023.01.058
Sun L, Wang L, Qian Y et al (2019) Feature selection using lebesgue and entropy measures for incomplete neighborhood decision systems. Knowl-Based Syst 186:104942. https://doi.org/10.1016/j.knosys.2019.104942
Chen Y, Zhang Z, Zheng J et al (2017) Gene selection for tumor classification using neighborhood rough sets and entropy measures. J Biomed Inform 67:59–68. https://doi.org/10.1016/j.jbi.2017.02.007
Yang D, Cai M, Li Q et al (2022) Multigranulation fuzzy probabilistic rough set model on two universes. Int J Approx Reason 145:18–35. https://doi.org/10.1016/j.ijar.2022.03.002
Hu Q, Zhang L, An S et al (2011) On robust fuzzy rough set models. IEEE Trans Fuzzy Syst 20(4):636–651. https://doi.org/10.1109/TFUZZ.2011.2181180
Yang C, Liu H, McLoone S et al (2017) A novel variable precision reduction approach to comprehensive knowledge systems. IEEE Trans Cybern 48(2):661–674. https://doi.org/10.1109/TCYB.2017.2648824
Liu K, Yang X, Yu H et al (2019) Rough set based semi-supervised feature selection via ensemble selector. Knowl-Based Syst 165:282–296. https://doi.org/10.1016/j.knosys.2018.11.034
Zhao S, Tsang E, Chen D (2009) The model of fuzzy variable precision rough sets. IEEE Trans Fuzzy Syst 17(2):451–467. https://doi.org/10.1109/TFUZZ.2009.2013204
Hu M, Guo Y, Chen D et al (2023) Attribute reduction based on neighborhood constrained fuzzy rough sets. Knowl-Based Syst 274:110632. https://doi.org/10.1016/j.knosys.2023.110632
Ziarko W (1993) Variable precision rough set model. J Comput Syst Sci 46(1):39–59. https://doi.org/10.1016/0022-0000(93)90048-2
Hu Q, An S, Yu D (2010) Soft fuzzy rough sets for robust feature evaluation and selection. Inf Sci 180(22):4384–4400. https://doi.org/10.1016/j.ins.2010.07.010
Hamidzadeh J, Rezaeenik E, Moradi M (2021) Predicting users’ preferences by fuzzy rough set quarter-sphere support vector machine. Appl Soft Comput 112:107740. https://doi.org/10.1016/j.asoc.2021.107740
An S, Hu Q, Yu D et al (2012) Soft minimum-enclosing-ball based robust fuzzy rough sets. Fund Inform 115(2–3):189–202. https://doi.org/10.3233/FI-2012-649
Hu Q, Liu J, Yu D (2008) Mixed feature selection based on granulation and approximation. Knowl-Based Syst 21(4):294–304. https://doi.org/10.1016/j.knosys.2007.07.001
Yang X, Chen H, Li T et al (2022) A noise-aware fuzzy rough set approach for feature selection. Knowl-Based Syst 250:109092. https://doi.org/10.1016/j.knosys.2022.109092
Liu J, Lin Y, Ding W et al (2022) Fuzzy mutual information-based multilabel feature selection with label dependency and streaming labels. IEEE Trans Fuzzy Syst 31(1):77–91. https://doi.org/10.1109/TFUZZ.2022.3182441
Moslemnejad S, Hamidzadeh J (2021) Weighted support vector machine using fuzzy rough set theory. Soft Comput 25(13):8461–8481. https://doi.org/10.1007/s00500-021-05773-7
An S, Zhao E, Wang C et al (2021) Relative fuzzy rough approximations for feature selection and classification. IEEE Transactions on Cybernetics 53(4):2200–2210. https://doi.org/10.1109/TCYB.2021.3112674
An S, Zhang M, Wang C et al (2023) Robust fuzzy rough approximations with knn granules for semi-supervised feature selection. Fuzzy Sets Syst 461:108476. https://doi.org/10.1016/j.fss.2023.01.011
Xu J, Meng X, Qu K et al (2023) Feature selection using relative dependency complement mutual information in fitting fuzzy rough set model. Appl Intell 1–24. https://doi.org/10.1007/s10489-022-04445-9
Liang P, Lei D, Chin K et al (2022) Feature selection based on robust fuzzy rough sets using kernel-based similarity and relative classification uncertainty measures. Knowl-Based Syst 255:109795. https://doi.org/10.1016/j.knosys.2022.109795
Mahajan S, Gupta S (2021) On fully intuitionistic fuzzy multiobjective transportation problems using different membership functions. Ann Oper Res 296:211–241. https://doi.org/10.1007/s10479-019-03318-8
Li S (2012) Some comments on fuzzy variables with different membership functions. Soft Comput 16:505–509. https://doi.org/10.1007/s00500-011-0749-5
Kim S, Lee M, Lee J (2017) A study of fuzzy membership functions for dependence decision-making in security robot system. Neural Comput Appl 28:155–164. https://doi.org/10.1007/s00521-015-2044-3
Zeng X, Singh M (1996) A relationship between membership functions and approximation accuracy in fuzzy systems. IEEE Trans Syst Man Cybern B Cybern 26(1):176–180. https://doi.org/10.1109/3477.484451
Jiang X, Yi Z, Lv J (2006) Fuzzy svm with a new fuzzy membership function. Neural Comput Appl 15:268–276. https://doi.org/10.1007/s00521-006-0028-z
Deng J, Deng Y (2021) Information volume of fuzzy membership function. Int J Comput Commun Control 16(1). https://doi.org/10.15837/ijccc.2021.1.4106
Li Y, Wei S, Liu X et al (2021) A novel robust fuzzy rough set model for feature selection. Complexity 2021:1–12. https://doi.org/10.1155/2021/6685396
Yuan M, Xu J, Li T et al (2023) Feature selection based on self-information and entropy measures for incomplete neighborhood decision systems. Complex Intell Syst 9(2):1773–1790. https://doi.org/10.1007/s40747-022-00882-8
Lin Y, Liu H, Zhao H et al (2022) Hierarchical feature selection based on label distribution learning. IEEE Trans Knowl Data Eng 35(6):5964–5976. https://doi.org/10.1109/TKDE.2022.3177246
Theerens A, Cornelis C (2023) Fuzzy rough sets based on fuzzy quantification. Fuzzy Sets Syst 473:108704. https://doi.org/10.1016/j.fss.2023.108704
Wang G (2003) Rough reduction in algebra view and information view. Int J Intell Syst 18(6):679–688. https://doi.org/10.1002/int.10109
Qu K, Xu J, Han Z et al (2023) Maximum relevance minimum redundancy-based feature selection using rough mutual information in adaptive neighborhood rough sets. Appl Intell 1–20. https://doi.org/10.1007/s10489-022-04398-z
Gou H, Zhang X, Yang J et al (2023) Three-way fusion measures and three-level feature selections based on neighborhood decision systems. Appl Soft Comput 148:110842. https://doi.org/10.1016/j.asoc.2023.110842
Xu J, Sun Y, Qu K et al (2022) Online group streaming feature selection using entropy-based uncertainty measures for fuzzy neighborhood rough sets. Complex Intell Syst 8(6):5309–5328. https://doi.org/10.1007/s40747-022-00763-0
Yuan Z, Chen H, Xie P et al (2021) Attribute reduction methods in fuzzy rough set theory: An overview, comparative experiments, and new directions. Appl Soft Comput 107:107353. https://doi.org/10.1016/j.asoc.2021.107353
Wang G, Yu H, Yang D et al (2002) Decision table reduction based on conditional information entropy. Chin J Comput 25(7):759–766. https://doi.org/10.3321/j.issn:0254-4164.2002.07013
Wu Y, Ianakiev K, Govindaraju V (2002) Improved k-nearest neighbor classification. Pattern Recogn 35(10):2311–2318. https://doi.org/10.1016/S0031-3203(01)00132-7
Breiman L, Friedman J, Olshen R et al (1984) Classification and regression trees (cart). Biometrics 40(3):358. https://doi.org/10.2307/2530946
Cortes C, Vapnik V (1995) Support-vector networks. Mach Learn 20:273–297. https://doi.org/10.1023/A:1022627411411
Rennie JD, Shih L, Teevan J, et al (2003) Tackling the poor assumptions of naive bayes text classifiers. In: Proceedings of the 20th international conference on machine learning (ICML-03). pp 616–623
Sun L, Wang L, Ding W et al (2020) Feature selection using fuzzy neighborhood entropy-based uncertainty measures for fuzzy neighborhood multigranulation rough sets. IEEE Trans Fuzzy Syst 29(1):19–33. https://doi.org/10.1109/TFUZZ.2020.2989098
Qu K, Xu J, Hou Q et al (2023) Feature selection using information gain and decision information in neighborhood decision system. Appl Soft Comput 136:110100. https://doi.org/10.1016/j.asoc.2023.110100
Tan A, Wu W, Qian Y et al (2018) Intuitionistic fuzzy rough set-based granular structures and attribute subset selection. IEEE Trans Fuzzy Syst 27(3):527–539. https://doi.org/10.1109/TFUZZ.2018.2862870
Xu J, Wang Y, Mu H et al (2019) Feature genes selection based on fuzzy neighborhood conditional entropy. J Intell Fuzz Syst 36(1):117–126. https://doi.org/10.3233/JIFS-18100
Xu J, Qu K, Meng X et al (2022) Feature selection based on multiview entropy measures in multiperspective rough set. Int J Intell Syst 37(10):7200–7234. https://doi.org/10.1002/int.22878
Xu F, Miao D, Wei L (2009) Fuzzy-rough attribute reduction via mutual information with an application to cancer classification. Comput Math Appl 57(6):1010–1017. https://doi.org/10.1016/j.camwa.2008.10.027
Sun L, Zhang X, Qian Y et al (2019) Feature selection using neighborhood entropy-based uncertainty measures for gene expression data classification. Inf Sci 502:18–41. https://doi.org/10.1016/j.ins.2019.05.072
Qian Y, Wang Q, Cheng H et al (2015) Fuzzy-rough feature selection accelerator. Fuzzy Sets Syst 258:61–78. https://doi.org/10.1016/j.fss.2014.04.029
Sun L, Li M, Ding W et al (2022) Afnfs: Adaptive fuzzy neighborhood-based feature selection with adaptive synthetic over-sampling for imbalanced data. Inf Sci 612:724–744. https://doi.org/10.1016/j.ins.2022.08.118
Wang C, Huang Y, Ding W et al (2021) Attribute reduction with fuzzy rough self-information measures. Inf Sci 549:68–86. https://doi.org/10.1016/j.ins.2020.11.021
Hu Q, Yu D, Pedrycz W et al (2010) Kernelized fuzzy rough sets and their applications. IEEE Trans Knowl Data Eng 23(11):1649–1667. https://doi.org/10.1109/TKDE.2010.260
Priya R, Sivaraj R (2017) Dynamic genetic algorithm-based feature selection and incomplete value imputation for microarray classification. Curr Sci 126–131. https://doi.org/10.18520/cs/v112/i01/126-131
Priya R, Kuppuswami S (2012) A genetic algorithm based approach for imputing missing discrete attribute values in databases. WSEAS Trans Inf Sci Appl 9(6):169–178
Friedman M (1940) A comparison of alternative tests of significance for the problem of m rankings. Ann Math Stat 11(1):86–92
Dunn O (1961) Multiple comparisons among means. J Am Stat Assoc 56(293):52–64. https://doi.org/10.1080/01621459.1961.10482090
Acknowledgements
The paper is supported in part by the National Natural Science Foundation of China under Grant (61976082, 62002103).
Author information
Authors and Affiliations
Contributions
Conceptualization: Jiucheng Xu, Shan Zhang; Methodology: Shan Zhang; Writing - original draft preparation: Shan Zhang, Miaoxian Ma, Wulin Niu; Writing - review and editing: Shan Zhang, Jianghao Duan; Funding acquisition: Jiucheng Xu.
Corresponding author
Ethics declarations
Ethical and informed consent for data used
The data used in this paper do not involve ethical issues.
Competing Interests
The authors have no competing interests to declare that are relevant to the content of this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendix A
Appendix A
In this section, we supplement certain figures from the main text. Figures 11 and 12 are radar charts plotted based on the classification accuracy of the algorithm across different datasets and classifiers. Tables 20, 21, 22, and 23 provide the rankings of the algorithm’s classification accuracy on various classifiers.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Xu, J., Zhang, S., Ma, M. et al. An attribute reduction algorithm using relative decision mutual information in fuzzy neighborhood decision system. Appl Intell 55, 217 (2025). https://doi.org/10.1007/s10489-024-06171-w
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06171-w