Abstract
Similar to single-label learning, multi-label learning employs feature selection technique to alleviate the curse of dimensionality. Many multi-label methods, which utilize label correlation or instance correlation to select meaningful features, were proposed in recent years. However, these multi-label feature selection methods explored the label correlation or instance correlation via similarity measures, which may not perform well in revealing complex relationships between labels and instances. Furthermore, label correlation and instance correlation are employed as independent strategy to select the discriminative features, and no general framework can currently be considered the two together as to their effect. In this paper, we propose a new multi-label feature selection method named CMFSS, which explicitly explores the label correlation and instance correlation in a collaborative manner. Firstly, CMFSS learns the label correlation and the instance correlation via the ADMM technique. Secondly, the learned label correlation and instance correlation are seamlessly incorporated into the multi-label feature selection model. Finally, CMFSS utilizes \(\ell _{2,1}\)-norm as sparsity regularization to control the model complexity. Extensive empirical evaluations conducted on multiple benchmark datasets clearly show the superiority of the proposed multi-label feature selection method.











Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availibility
The datasets employed during this study are available in the Mulan Library: http://mulan.sourceforge.net/datasets-mlc.html.
References
Zhang M-L, Zhou Z-H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837
Huang Z-A, Zhang J, Zhu Z, Wu EQ, Tan KC (2020) Identification of autistic risk candidate genes and toxic chemicals via multilabel learning. IEEE Trans Neur Netw Learn Sys 32(9):3971–3984
Zhang J, Li S, Jiang M, Tan KC (2020) Learning from weakly labeled data based on manifold regularized sparse model. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.3015269
Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Patt Recognit 37(9):1757–1771
Dai L, Zhang J, Li C, Zhou C, Li S (2019) Multi-label feature selection with application to TCM state identification. Concurr Comput: Pract Exper 31(23):4634
Zhang J, Lin Y, Jiang M, Li S, Tang Y, Tan K.C (2020) Multi-label feature selection via global relevance and redundancy optimization. In: Proc. 29th Int. Joint Conf. Artif. Intell, pp. 2512–2518
Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Patt Recognit 104:107344
Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with local discriminant model and label correlations. Neurocomputing 442:98–115
Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. IJCAI 16:1627–33
Huang R, Jiang W, Sun G (2018) Manifold-based constraint laplacian score for multi-label feature selection. Patt Recognit Lett 112:346–352
Cai Z, Zhu W (2017) Feature selection for multi-label classification using neighborhood preservation. IEEE/CAA J Automat Sinica 5(1):320–330
Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with constraint regression and adaptive spectral graph. Knowl-Bas Sys 212:106621
Zhang M-L, Zhou Z-H (2007) Ml-knn: a lazy learning approach to multi-label learning. Patt Recognit 40(7):2038–2048
Zhang M-L, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351
Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16–17):1897–1916
Che X, Chen D, Mi J (2020) A novel approach for learning label correlation with application to feature selection of multi-label data. Infor Sci 512:795–812
Wu X, Jiang B, Yu K, Chen H, Miao C (2020) Multi-label causal feature selection. In: Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 6430–6437
Wang H, Chen C, Liu W, Chen K, Hu T, Chen G (2020) Incorporating label embedding and feature augmentation for multi-dimensional classification. In: Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 6178–6185
Zhu Y, Kwok JT, Zhou Z-H (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094
Feng L, An B, He S (2019) Collaboration based multi-label learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 3550–3557
Dai L, Du G, Zhang J, Li C, Wei R, Li S (2020) Joint multilabel classification and feature selection based on deep canonical correlation analysis. Concurr Comput: Pract Exper 32(22):5864
Du G, Zhang J, Luo Z, Ma F, Ma L, Li S (2020) Joint imbalanced classification and feature selection for hospital readmissions. Knowl-Based Sys 200:106020
Lee J, Kim D-W (2015) Fast multi-label feature selection based on information-theoretic feature ranking. Patt Recognit 48(9):2761–2771
Lee J, Kim D-W (2015) Memetic feature selection algorithm for multi-label classification. Infor Sci 293:80–96
Zhang M-L, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Infor Sci 179(19):3218–3229
Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2020) Mfs-mcdm: multi-label feature selection using multi-criteria decision making. Knowl-Based Sys 206:106365
Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2021) An efficient pareto-based feature selection algorithm for multi-label classification. Infor Sci 581:428–447
Sun Z, Zhang J, Dai L, Li C, Zhou C, Xin J, Li S (2019) Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing 329:447–456
Nie F, Huang H, Cai X, Ding C.H.Q (2010) Efficient and robust feature selection via joint 2, 1-norms minimization. In: 24th Annual conference on neural information processing systems, Vancouver, British Columbia, Canada, pp. 1813–1821
Chen S-B, Zhang Y-M, Ding CH, Zhang J, Luo B (2019) Extended adaptive lasso for multi-class and multi-label feature selection. Knowl-Based Sys 173:28–36
Fan Y, Chen B, Huang W, Liu J, Weng W, Lan W (2022) Multi-label feature selection based on label correlations and feature redundancy. Knowl-Based Sys 241:108256
Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc
Zhang Q.-W, Zhong Y, Zhang M.-L (2018) Feature-induced labeling information enrichment for multi-label learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 32, pp. 4446–4453 (2018)
Bartels RH, Stewart GW (1972) Solution of the matrix equation ax+ xb= c [f4]. Commun ACM 15(9):820–826
Liu Y, Wen K, Gao Q, Gao X, Nie F (2018) Svm based multi-label learning with missing labels for image annotation. Patt Recognit 78:307–317
Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Sys 159:148–157
Yeh C.-K, Wu W.-C, Ko W.-J, Wang Y.-C.F (2017) Learning deep latent space for multi-label classification. In: Thirty-first AAAI conference on artificial intelligence, pp. 2838–2844
Zhang J, Li C, Sun Z, Luo Z, Zhou C, Li S (2019) Towards a unified multi-source-based optimization framework for multi-label learning. Appl Soft Comput 76:425–435
Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103
Lee J, Kim D-W (2017) Scls: multi-label feature selection based on scalable criterion for large label set. Patt Recognit 66:342–352
Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Patt Recognit 95:136–150
Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30
Du G, Zhang J, Jiang M, Long J, Lin Y, Li S, Tan KC (2021) Graph-based class-imbalance learning with label enhancement. IEEE Trans Neur Netw Learn Sys. https://doi.org/10.1109/TNNLS.2021.3133262
Zhang P, Liu G, Li S (2019) Distinguishing two types of labels for multi-label feature selection. Patt Recognit 95:72–82
Yu K, Cai M, Li J (2021) Multilabel feature selection: a local causal structure learning approach. IIEEE Trans Neur Netw Learn Sys. https://doi.org/10.1109/TNNLS.2021.3111288
Hou S, Chu Y, Fei J (2020) Intelligent global sliding mode control using recurrent feature selection neural network for active power filter. IEEE Trans Ind Electr 68(8):7320–7329
Liu J, Lin Y, Du J, Zhang H, Chen Z, Zhang J (2022) ASFS: a novel streaming feature selection for multi-label data based on neighborhood rough set. Appl Intell. https://doi.org/10.1007/s10489-022-03366-x
Tan A, Liang J, Wu W, Zhang J (2022) Semi-supervised partial multi-label classification via consistency learning. Patt Recognit. https://doi.org/10.1016/j.patcog.2022.108839
Zhang J, Wu H, Jiang M, Liu J, Li S, Tang Y, Long J (2022) Group-preserving label-specific feature selection for multi-label learning. Exp Sys Appl. https://doi.org/10.1016/j.eswa.2022.118861
Huang M, Zhuang F, Zhang X, Ao X, Niu Z, Zhang M, He Q (2019) Supervised representation learning for multi-label classification. Mach Learn 108(5):747–763
Xiong Y, Wang L, Wang Q, Liu S, Kou B (2022) Improved convolutional neural network with feature selection for imbalanced ECG Multi-Factor classification. Measurement 189:110471
Cui Z, Lu N (2021) Feature selection accelerated convolutional neural networks for visual tracking. Appl Intell 51(11):8230–8244
CheLe T, Kim Y, Kim H (2019) Network intrusion detection based on novel feature selection model and various recurrent neural networks. Appl Sci 9(7):1392
Liu X, Zhang H, Kong X, Lee K (2020) Wind speed forecasting using deep neural network with feature selection. Neurocomputing 397:393–403
Acknowledgements
This work was supported in part by the National Nature Science Foundation of China (Nos. 61876159, 61806172, 62076116, 62106084 & U1705286), the China Postdoctoral Science Foundation Grant No. (2019M652257), the National Natural Science Foundation of Guangdong, China (No. 2022A1515010468), the Fundamental Research Funds for the Central Universities, Jinnan University (No. 21621026), the Science and Technology Project in Guangzhou (No. 202201010498), Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Informatization (No. 2021B1212040007), Startup Foundation for Introducing Talent of Yanshan University (No. 8190550).
Author information
Authors and Affiliations
Corresponding authors
Ethics declarations
Conflict of interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Dai, L., Zhang, J., Du, G. et al. Toward embedding-based multi-label feature selection with label and feature collaboration. Neural Comput & Applic 35, 4643–4665 (2023). https://doi.org/10.1007/s00521-022-07924-9
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-022-07924-9