Abstract
Information-theoretic measures have been commonly applied to evaluate the relevance and redundancy in multi-label feature selection. However, the current multi-label feature selection methods based on information-theoretic measures neglect the dynamic changes in the relevance of selected features and candidate features. Furthermore, they also do not fully consider the influence of label redundancy on the relevance of candidate features. In this paper, we first propose a new feature relevance term named Dynamic Correlation Change (DCC), which uses two conditional mutual information terms to evaluate the dynamic changes in the relevance of selected features and candidate features. We then introduce a new label redundancy term named Label Redundancy with Interaction Information (LRII), which more accurately quantifies the influence of label redundancy on the relevance of candidate features. On this basis, we design a new multi-label feature selection method, called Maximum Dynamic Correlation Change and Minimum Label Redundancy (MDCCMLR), by combining DCC and LRII. Finally, we conduct extensive experiments in order to verify the performance of our method by comparing it with some state-of-the-art multi-label feature selection methods based on information-theoretic measures in terms of six evaluation metrics. The experimental results show that the MDCCMLR method outperforms the other comparison methods on all six evaluation metrics.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Cai Z, Zhu W (2018) Multi-label feature selection via feature manifold learning and sparsity regularization. Int J mach learn cyber 9:1321–1334. https://doi.org/10.1007/s13042-017-0647-y
Chandrashekar G, Sahin F (2014) A survey on feature selection methods. Comput Elect Eng 40(1):16–28. https://doi.org/10.1016/j.compeleceng.2013.11.024
Che X, Chen D, Mi J (2022) Label correlation in multi-label classification using local attribute reductions with fuzzy rough sets. Fuzzy Sets Syst 426:121–144. https://doi.org/10.1016/j.fss.2021.03.016
Cover TM (1999) Elements Inform Theory. John Wiley & Sons
Davis JJ, Wattam AR, Aziz RK et al (2020) The patric bioinformatics resource center: expanding data and analysis capabilities. Nucleic Acids Res 48(D1):D606–D612. https://doi.org/10.1093/nar/gkz943
Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. The Journal of Machine Learning Research 7:1–30. http://jmlr.org/papers/v7/demsar06a.html
Fan Y, Chen B, Huang W et al (2022) Multi-label feature selection based on label correlations and feature redundancy. Knowled-Based Sys. https://doi.org/10.1016/j.knosys.2022.108256
Gao W, Hao P, Wu Y et al (2023) A unified low-order information-theoretic feature selection framework for multi-label learning. Pattern Recogn 134(109):111. https://doi.org/10.1016/j.patcog.2022.109111
Gonzalez-Lopez J, Ventura S, Cano A (2020a) Distributed multi-label feature selection using individual mutual information measures. Knowledge-Based Systems 188. https://doi.org/10.1016/j.knosys.2019.105052
Gonzalez-Lopez J, Ventura S, Cano A (2020b) Distributed selection of continuous features in multilabel classification using mutual information. IEEE Trans Neural Networks Learn Syst 31(7):2280–2293. https://doi.org/10.1109/TNNLS.2019.2944298
Gulcan EB, Can F (2022) Unsupervised concept drift detection for multi-label data streams. Artificial Intellig Rev. https://doi.org/10.1007/s10462-022-10232-2
Hall MA, Frank E, Holmes G et al (2009) The WEKA data mining software: an update. ACM SIGKDD Explorations Newsl 11(1):10–18. https://doi.org/10.1145/1656274.1656278
Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2020a) MFS-MCDM: Multi-label feature selection using multi-criteria decision making. Knowl-Based Syst 206(106):365. https://doi.org/10.1016/j.knosys.2020.106365
Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2020b) MGFS: A multi-label graph-based feature selection algorithm via pagerank centrality. Expert Syst Appl 142(113):024. https://doi.org/10.1016/j.eswa.2019.113024
Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2021) A bipartite matching-based feature selection for multi-label learning. Int J Mach Learn Cybern 12:459–475. https://doi.org/10.1007/s13042-020-01180-w
Hu L, Gao L, Li Y et al (2022) Feature-specific mutual information variation for multi-label feature selection. Inf Sci 593:449–471. https://doi.org/10.1016/j.ins.2022.02.024
Huang X, Chen B, Xiao L et al (2021) Label-aware document representation via hybrid attention for extreme multi-label text classification. Neural Process Lett. https://doi.org/10.1007/s11063-021-10444-7
Kashef S, Nezamabadi-pour H, Nikpour B (2018) Multilabel Feat Select: Comprehensive Rev Guiding Exp. Data Mining and Knowledge Discovery, Wiley Interdisciplinary Reviews. https://doi.org/10.1002/widm.1240
Lee J, Kim D (2013) Feature selection for multi-label classification using multivariate mutual information. Pattern Recogn Lett 34(3):349–357. https://doi.org/10.1016/j.patrec.2012.10.005
Lee J, Kim D (2015) Mutual information-based multi-label feature selection using interaction information. Expert Syst Appl 42(4):2013–2025. https://doi.org/10.1016/j.eswa.2014.09.063
Lee J, Kim D (2017) SCLS: multi-label feature selection based on scalable criterion for large label set. Pattern Recogn 66:342–352. https://doi.org/10.1016/j.patcog.2017.01.014
Li L, Liu H, Ma Z, et al (2014) Multi-label feature selection via information gain. In: Luo X, Yu JX, Li Z (eds) Advanced Data Mining and Applications - 10th International Conference, ADMA 2014, Guilin, China, December 19-21, 2014. Proceedings, Lecture Notes in Computer Science, vol 8933. Springer, pp 345–355, https://doi.org/10.1007/978-3-319-14717-8_27
Lin Y, Hu Q, Liu J et al (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103. https://doi.org/10.1016/j.neucom.2015.06.010
Lin Y, Hu Q, Liu J et al (2016) Multi-label feature selection based on neighborhood mutual information. Appl Soft Comput 38:244–256. https://doi.org/10.1016/j.asoc.2015.10.009
Lin Y, Li Y, Wang C et al (2018) Attribute reduction for multi-label learning with fuzzy rough set. Knowl-Based Syst 152:51–61. https://doi.org/10.1016/j.knosys.2018.04.004
Ma XA, Ju C (2022) Fuzzy information-theoretic feature selection via relevance, redundancy, and complementarity criteria. Inf Sci 611:564–590. https://doi.org/10.1016/j.ins.2022.08.067
Ma XA, Yao Y (2018) Three-way decision perspectives on class-specific attribute reducts. Inf Sci 450:227–245. https://doi.org/10.1016/j.ins.2018.03.049
Ma XA, Xu H, Ju C (2023) Class-specific feature selection via maximal dynamic correlation change and minimal redundancy. Expert Syst Appl 229(120):455. https://doi.org/10.1016/j.eswa.2023.120455
Mariello A, Battiti R (2018) Feature selection based on the neighborhood entropy. IEEE Transa Neural Net Learn Syst 29(12):6313–6322. https://doi.org/10.1109/TNNLS.2018.2830700
Pereira RB, Plastino A, Zadrozny B et al (2018) Categorizing feature selection methods for multi-label classification. Artif Intell Rev 49(1):57–78. https://doi.org/10.1007/s10462-016-9516-4
Qian W, Long X, Wang Y et al (2020) Multi-label feature selection based on label distribution and feature complementarity. Appl Soft Comput 90(106):167. https://doi.org/10.1016/j.asoc.2020.106167
Qian W, Xiong C, Wang Y (2021) A ranking-based feature selection for multi-label classification with fuzzy relative discernibility. Appl Soft Comput 102(106):995. https://doi.org/10.1016/j.asoc.2020.106995
Read J, Pfahringer B, Holmes G (2008) Multi-label classification using ensembles of pruned sets. In: Proceedings of the 8th IEEE International Conference on Data Mining (ICDM 2008), December 15-19, 2008, Pisa, Italy, IEEE. IEEE Computer Society, pp 995–1000, https://doi.org/10.1109/ICDM.2008.74
Sun Z, Zhang J, Dai L et al (2019) Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing 329:447–456. https://doi.org/10.1016/j.neucom.2018.10.047
Trohidis K, Tsoumakas G, Kalliris G et al (2008) (2008) Multi-label classification of music into emotions. In: Bello JP, Chew E, Turnbull D (eds) ISMIR 2008, 9th International Conference on Music Information Retrieval. Drexel University, Philadelphia, USA, pp 325–330
Tsoumakas G, Xioufis ES, Vilcek J, et al (2011) MULAN: A java library for multi-label learning. The Journal of Machine Learning Research. http://dl.acm.org/citation.cfm?id=2021078
Wang H, Ding Y, Tang J et al (2021a) Identify RNA-associated subcellular localizations based on multi-label learning using chou’s 5-steps rule. BMC Genomics 22(1):1–14. https://doi.org/10.1186/s12864-020-07347-7
Wang R, Kwong S, Wang X et al (2021b) Active \(k\)-labelsets ensemble for multi-label classification. Pattern Recogn 109(107):583. https://doi.org/10.1016/j.patcog.2020.107583
Wu JS, Huang SJ, Zhou ZH (2014) Genome-wide protein function prediction through multi-instance multi-label learning. IEEE/ACM Trans Comput Biol Bioinf 11(5):891–902. https://doi.org/10.1109/TCBB.2014.2323058
Xia Y, Chen K, Yang Y (2021) Multi-label classification with weighted classifier selection and stacked ensemble. Inf Sci 557:421–442. https://doi.org/10.1016/j.ins.2020.06.017
Xiao Y, Li Y, Yuan J et al (2021) History-based attention in seq2seq model for multi-label text classification. Knowl-Based Syst 224(107):094. https://doi.org/10.1016/j.knosys.2021.107094
Yu G, Rangwala H, Domeniconi C et al (2013) Protein function prediction using multilabel ensemble classification. IEEE ACM Trans Comput Biol Bioinformat 10(4):1045–1057. https://doi.org/10.1109/TCBB.2013.111
Zhang J, Wu H, Jiang M et al (2023a) Group-preserving label-specific feature selection for multi-label learning. Expert Syst Appl 213(118):861. https://doi.org/10.1016/j.eswa.2022.118861
Zhang M, Zhou Z (2007) ML-KNN: A lazy learning approach to multi-label learning. Pattern Recogn 40(7):2038–2048. https://doi.org/10.1016/j.patcog.2006.12.019
Zhang M, Zhou Z (2014) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837. https://doi.org/10.1109/TKDE.2013.39
Zhang P, Gao W (2021) Feature relevance term variation for multi-label feature selection. Appl Intell 51(7):5095–5110. https://doi.org/10.1007/s10489-020-02129-w
Zhang P, Liu G, Gao W (2019) Distinguishing two types of labels for multi-label feature selection. Pattern Recogn 95:72–82. https://doi.org/10.1016/j.patcog.2019.06.004
Zhang P, Liu G, Gao W et al (2021a) Multi-label feature selection considering label supplementation. Pattern Recogn 120(108):137. https://doi.org/10.1016/j.patcog.2021.108137
Zhang P, Sheng J, Gao W et al (2022) Multi-label feature selection method based on dynamic weight. Soft Comput 26:2793–2805. https://doi.org/10.1007/s00500-021-06664-7
Zhang P, Liu G, Song J (2023b) Mfsjmi: Multi-label feature selection considering join mutual information and interaction weight. Pattern Recogn 138(109):378. https://doi.org/10.1016/j.patcog.2023.109378
Zhang X, Yao H, Lv Z et al (2021b) Class-specific information measures and attribute reducts for hierarchy and systematicness. Inf Sci 563:196–225. https://doi.org/10.1016/j.ins.2021.01.080
Zhao D, Gao Q, Lu Y et al (2022) Learning multi-label label-specific features via global and local label correlations. Soft Comput 26:2225–2239. https://doi.org/10.1007/s00500-021-06645-w
Acknowledgements
This work was supported by the Zhejiang Provincial Natural Science Foundation of China (Grant No. LY21F020008), the Fundamental Research Funds for the Provincial Universities of Zhejiang (Grant No. XT202311), the Natural Science Foundation of Chongqing of China (Grant Nos. cstc2021jcyj-msxmX0654, cstc2021jcyj-msxmX0495), the Key Science and Technology Research Program of Chongqing Municipal Education Commission of China (Grant No. KJZD-K202101305), and the Yingcai Program of Chongqing of China (Grant No. cstc2021ycjh-bgzxm0218). We would like to express our gratitude to the reviewers for their constructive comments and suggestions, which have greatly improved the quality of this paper. We would also like to acknowledge the experimental work carried out by graduate student Haibo Liu during the subsequent revisions of this paper.
Author information
Authors and Affiliations
Contributions
Xi-Ao Ma wrote the manuscript, conceived of the presented idea and designed the experiments. Wentian Jiang carried out the experiments. Yun Ling and Bailin Yang supervised the project. All authors approved the manuscript.
Corresponding author
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Ma, XA., Jiang, W., Ling, Y. et al. Multi-label feature selection via maximum dynamic correlation change and minimum label redundancy. Artif Intell Rev 56 (Suppl 3), 3099–3142 (2023). https://doi.org/10.1007/s10462-023-10599-w
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-023-10599-w