Abstract
Traditional information theory-based feature selection methods are designed for discrete features, which require additional discretization steps when working with continuous features. In contrast, fuzzy information theory-based feature selection methods can handle continuous features directly. However, most existing fuzzy information theory-based feature selection methods do not consider the dynamic interaction between candidate features and the already selected ones. To address this issue, we propose a dynamic weighted feature selection method based on fuzzy interaction information that can handle continuous features. First, we use fuzzy information theory metrics to characterize the concepts of feature relevance, redundancy, and interaction. Second, we define a fuzzy interaction weight factor that can quantify the redundancy and interaction between features by using fuzzy interaction information. Third, we design a novel feature selection algorithm called fuzzy dynamic interactive weighted feature selection (FDIWFS) by combining the fuzzy interaction weight factor with a sequential forward search strategy. To evaluate the effectiveness of FDIWFS, we compare it with eight state-of-the-art feature selection methods on fifteen publicly available datasets. The results of comparative experiments demonstrate that FDIWFS outperforms the other methods in terms of classification performance.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data Availability
The data used in this article are publicly available online and have been cited in the paper.
References
Abdelrazek M, Abd Elaziz M, El-Baz A (2024) CDMO: Chaotic dwarf mongoose optimization algorithm for feature selection. Sci Rep 14(1):701. https://doi.org/10.1038/s41598-023-50959-8
Chen B, Zhang X, Yang J (2023) Feature selections based on three improved condition entropies and one new similarity degree in interval-valued decision systems. Eng Appl Artif Intell 126:107165. https://doi.org/10.1016/j.engappai.2023.107165
Dai J, Chen J (2020) Feature selection via normative fuzzy information weight with application into tumor classification. Appl Soft Comput 92:106299. https://doi.org/10.1016/j.asoc.2020.106299
Dai J, Xu Q (2013) Attribute selection based on information gain ratio in fuzzy rough set theory with application to tumor classification. Appl Soft Comput 13(1):211–221. https://doi.org/10.1016/j.asoc.2012.07.029
Deng T, Huang Y, Yang G et al (2022) Pointwise mutual information sparsely embedded feature selection. Int J Approx Reason 151:251–270. https://doi.org/10.1016/j.ijar.2022.09.012
Dunn OJ (1961) Multiple comparisons among means. J Amer Statist Assoc 56(293):52–64. https://doi.org/10.1080/01621459.1961.10482090
Fleuret F (2004) Fast binary feature selection with conditional mutual information. J Mach Leran Res 5(4941):1531–1555
Friedman M (1940) A comparison of alternative tests of significance for the problem of \(m\) rankings. Ann Math Stat 11(1):86–92. https://doi.org/10.1214/aoms/1177731944
Gu X, Guo J, Li C et al (2021) A feature selection algorithm based on redundancy analysis and interaction weight. Appl Intell 51(4):2672–2686. https://doi.org/10.1007/s10489-020-01936-5
Gu X, Guo J, Xiao L et al (2022) Conditional mutual information-based feature selection algorithm for maximal relevance minimal redundancy. Appl Intell 52(2):1436–1447. https://doi.org/10.1007/s10489-021-02412-4
Hu L, Gao W, Zhao K et al (2018) Feature selection considering two types of feature relevancy and feature interdependency. Expert Syst Appl 93:423–434. https://doi.org/10.1016/j.eswa.2017.10.016
Hu Q, Yu D, Xie Z (2006) Information-preserving hybrid data reduction based on fuzzy-rough techniques. Pattern Recognit Lett 27(5):414–423. https://doi.org/10.1016/j.patrec.2005.09.004
Iman RL, Davenport JM (1980) Approximations of the critical region of the fbietkan statistic. Comm Statist 9(6):571–595. https://doi.org/10.1080/03610928008827904
John GH, Kohavi R, Pfleger K (1994) Irrelevant features and the subset selection problem. In: Machine learning proceedings 1994. Elsevier, pp 121–129. https://doi.org/10.1016/B978-1-55860-335-6.50023-4
Khaire UM, Dhanalakshmi R (2019) Stability of feature selection algorithm: A review. J King Saud Univ - Comput Inf Sci. https://doi.org/10.1016/j.jksuci.2019.06.012
Kuncheva LI (2007) A stability index for feature selection. In: In Proceedings of the 25th IASTED International Multi-Conference on Artificial Intelligence and Applications, pp 390–395
Li J, Cheng K, Wang S et al (2017) Feature selection: A data perspective. ACM Comput Surv 50(6):1–45. https://doi.org/10.1145/3136625
Li Z (2021) A new feature selection using dynamic interaction. Pattern Anal Appl 24(1):203–215. https://doi.org/10.1007/s10044-020-00916-2
Li Z (2022) A feature selection method using dynamic dependency and redundancy analysis. Arab J Sci Eng 47(8):10419–10433. https://doi.org/10.1007/s13369-022-06590-2
Luo L, Yang J, Zhang X et al (2024) Tri-level attribute reduction based on neighborhood rough sets. Appl Intell 54(5):3786–3807. https://doi.org/10.1007/s10489-024-05361-w
Ma XA, Ju C (2022) Fuzzy information-theoretic feature selection via relevance, redundancy, and complementarity criteria. Inform Sci 611:564–590. https://doi.org/10.1016/j.ins.2022.08.067
Ma XA, Yao Y (2018) Three-way decision perspectives on class-specific attribute reducts. Inform Sci 450:227–245. https://doi.org/10.1016/j.ins.2018.03.049
Ma XA, Zhao XR (2019) Cost-sensitive three-way class-specific attribute reduction. Int J Approx Reason 105:153–174. https://doi.org/10.1016/j.ijar.2018.11.014
Ma XA, Xu H, Ju C (2023) Class-specific feature selection via maximal dynamic correlation change and minimal redundancy. Expert Syst Appl 229:120455. https://doi.org/10.1016/j.eswa.2023.120455
Markelle Kelly KNRachel Longjohn (2024) The UCI machine learning repository. https://archive.ics.uci.edu
Naik AK, Kuppili V (2022) Dynamic relevance and interdependent feature selection for continuous data. Expert Syst Appl 191:116302. https://doi.org/10.1016/j.eswa.2021.116302
Pedregosa F, Varoquaux G, Gramfort A, et al (2011) Scikit-learn: Machine learning in python. J Mach Leran Res 12(85):2825–2830. http://jmlr.org/papers/v12/pedregosa11a.html
Peng H, Long F, Ding C (2005) Feature selection based on mutual information criteria of max-dependency, max-relevance, and min-redundancy. IEEE Trans Pattern Anal Mach Intell 27(8):1226–1238. https://doi.org/10.1109/tpami.2005.159
Qu L, He J, Zhang G et al (2022) Entropy measure for a fuzzy relation and its application in attribute reduction for heterogeneous data. Appl Soft Comput 118:108455. https://doi.org/10.1016/j.asoc.2022.108455
Ramey JA (2016) Datamicroarray: collection of data sets for classification https://github.com/ramhiser/datamicroarray
Salem OA, Liu F, Chen YPP et al (2021) Feature selection and threshold method based on fuzzy joint mutual information. Int J Approx Reason 132:107–126. https://doi.org/10.1016/j.ijar.2021.01.003
Salem OA, Liu F, Chen YPP et al (2022) Fuzzy joint mutual information feature selection based on ideal vector. Expert Syst Appl 193:116453. https://doi.org/10.1016/j.eswa.2021.116453
Sun L, Ma Y, Ding W et al (2024) LSFSR: Local label correlation-based sparse multilabel feature selection with feature redundancy. Inform Sci 667:120501. https://doi.org/10.1016/j.ins.2024.120501
Sun X, Liu Y, Xu M et al (2013) Feature selection using dynamic weights for classification. Knowl-Based Syst 37:541–549. https://doi.org/10.1016/j.knosys.2012.10.001
Wan J, Chen H, Li T et al (2021) Dynamic interaction feature selection based on fuzzy rough set. Inform Sci 581:891–911. https://doi.org/10.1016/j.ins.2021.10.026
Wan J, Chen H, Li T et al (2022) R2CI: Information theoretic-guided feature selection with multiple correlations. Pattern Recognit 127:108603. https://doi.org/10.1016/j.patcog.2022.108603
Wang C, Huang Y, Shao M et al (2019) Uncertainty measures for general fuzzy relations. Fuzzy Sets Syst 360:82–96. https://doi.org/10.1016/j.fss.2018.07.006
Wang C, Qian Y, Ding W et al (2021) Feature selection with fuzzy-rough minimum classification error criterion. IEEE Trans Fuzzy Syst 30(8):2930–2942. https://doi.org/10.1109/TFUZZ.2021.3097811
Wang C, Wang C, Qian Y, et al (2024) Feature selection based on weighted fuzzy rough sets. IEEE Trans Fuzzy Syst pp 1–11. https://doi.org/10.1109/TFUZZ.2024.3387571
Wang Z, Chen H, Yuan Z et al (2022) Exploiting fuzzy rough mutual information for feature selection. Appl Soft Comput 131:109769. https://doi.org/10.1016/j.asoc.2022.109769
Wollstadt P, Schmitt S, Wibral M (2023) A rigorous information-theoretic definition of redundancy and relevancy in feature selection based on (partial) information decomposition. J Mach Leran Res 24(131):1–44. http://jmlr.org/papers/v24/21-0482.html
Xia D, Wang G, Zhang Q et al (2023) Interactive fuzzy knowledge distance-guided attribute reduction with three-way accelerator. Knowl-Based Syst 279:110943. https://doi.org/10.1016/j.knosys.2023.110943
Xu J, Qu K, Sun Y et al (2023) Feature selection using self-information uncertainty measures in neighborhood information systems. Appl Intell 53(4):4524–4540. https://doi.org/10.1007/s10489-022-03760-5
Yu D, An S, Hu Q (2011) Fuzzy mutual information based min-redundancy and max-relevance heterogeneous feature selection. Int J Comput Intell Syst 4(4):619–633. https://doi.org/10.1080/18756891.2011.9727817
Yu L, Liu H (2004) Efficient feature selection via analysis of relevance and redundancy. J Mach Leran Res 5:1205–1224
Zaheer H, Rehman SU, Bashir M, et al (2024) A metaheuristic based filter-wrapper approach to feature selection for fake news detection. Multimed Tools Appl pp 1–30. https://doi.org/10.1007/s11042-024-18734-7
Zeng Z, Zhang H, Zhang R et al (2015) A novel feature selection method considering feature interaction. Pattern Recognit 48(8):2656–2666. https://doi.org/10.1016/j.patcog.2015.02.025
Zhang P, Gao W, Hu J et al (2021) A conditional-weight joint relevance metric for feature relevancy term. Eng Appl Artif Intell 106:104481. https://doi.org/10.1016/j.engappai.2021.104481
Zhang X, Yao Y (2022) Tri-level attribute reduction in rough set theory. Expert Syst Appl 190:116187. https://doi.org/10.1016/j.eswa.2021.116187
Zhao J, Wu D, Wu J et al (2024) Consistency approximation: Incremental feature selection based on fuzzy rough set theory. Pattern Recognit 155:110652. https://doi.org/10.1016/j.patcog.2024.110652
Acknowledgements
This work was supported by the Zhejiang Provincial Natural Science Foundation of China (Grant Nos. LY21F020008, LQ22G020003, LZ21F020004, LZ22F020008), the Natural Science Foundation of Chongqing of China (Grant Nos. cstc2021jcyj-msxmX0654, CSTB2024NSCQ-LZX0034), the Fundamental Research Funds for the Provincial Universities of Zhejiang, China (Grant No. XT202311), the National Nature Science Foundation of China (Grant Nos. U22A20102, 61976187), and the Major Project of Digital and Cutting-edge Disciplines Construction, Zhejiang Gongshang University, China (Grant Nos. SZJ2022A002, SZJ2022B007).
Author information
Authors and Affiliations
Contributions
Xi-Ao Ma: Conceptualization, Methodology, Writing - original draft preparation, Writing - review and editing, Funding acquisition, Supervision; Hao Xu: Methodology, Software, Validation, Yi Liu: Conceptualization, Methodology, Writing - review and editing, Funding acquisition.
Corresponding author
Ethics declarations
Competing interests
The authors declare that they have no conflicts of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
In the published paper, the incorrect corresponding author was captured.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ma, XA., Xu, H. & Liu, Y. Dynamic interactive weighted feature selection using fuzzy interaction information. Appl Intell 55, 166 (2025). https://doi.org/10.1007/s10489-024-06026-4
Accepted:
Published:
DOI: https://doi.org/10.1007/s10489-024-06026-4