Skip to main content
SpringerLink
Log in

Toward embedding-based multi-label feature selection with label and feature collaboration

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Similar to single-label learning, multi-label learning employs feature selection technique to alleviate the curse of dimensionality. Many multi-label methods, which utilize label correlation or instance correlation to select meaningful features, were proposed in recent years. However, these multi-label feature selection methods explored the label correlation or instance correlation via similarity measures, which may not perform well in revealing complex relationships between labels and instances. Furthermore, label correlation and instance correlation are employed as independent strategy to select the discriminative features, and no general framework can currently be considered the two together as to their effect. In this paper, we propose a new multi-label feature selection method named CMFSS, which explicitly explores the label correlation and instance correlation in a collaborative manner. Firstly, CMFSS learns the label correlation and the instance correlation via the ADMM technique. Secondly, the learned label correlation and instance correlation are seamlessly incorporated into the multi-label feature selection model. Finally, CMFSS utilizes \(\ell _{2,1}\)-norm as sparsity regularization to control the model complexity. Extensive empirical evaluations conducted on multiple benchmark datasets clearly show the superiority of the proposed multi-label feature selection method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Data availibility

The datasets employed during this study are available in the Mulan Library: http://mulan.sourceforge.net/datasets-mlc.html.

Notes

  1. http://mulan.sourceforge.net/datasets-mlc.html.

References

  1. Zhang M-L, Zhou Z-H (2013) A review on multi-label learning algorithms. IEEE Trans Knowl Data Eng 26(8):1819–1837

    Article  Google Scholar 

  2. Huang Z-A, Zhang J, Zhu Z, Wu EQ, Tan KC (2020) Identification of autistic risk candidate genes and toxic chemicals via multilabel learning. IEEE Trans Neur Netw Learn Sys 32(9):3971–3984

    Article  Google Scholar 

  3. Zhang J, Li S, Jiang M, Tan KC (2020) Learning from weakly labeled data based on manifold regularized sparse model. IEEE Trans Cybern. https://doi.org/10.1109/TCYB.2020.3015269

    Article  Google Scholar 

  4. Boutell MR, Luo J, Shen X, Brown CM (2004) Learning multi-label scene classification. Patt Recognit 37(9):1757–1771

    Article  Google Scholar 

  5. Dai L, Zhang J, Li C, Zhou C, Li S (2019) Multi-label feature selection with application to TCM state identification. Concurr Comput: Pract Exper 31(23):4634

    Article  Google Scholar 

  6. Zhang J, Lin Y, Jiang M, Li S, Tang Y, Tan K.C (2020) Multi-label feature selection via global relevance and redundancy optimization. In: Proc. 29th Int. Joint Conf. Artif. Intell, pp. 2512–2518

  7. Hu L, Li Y, Gao W, Zhang P, Hu J (2020) Multi-label feature selection with shared common mode. Patt Recognit 104:107344

    Article  Google Scholar 

  8. Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with local discriminant model and label correlations. Neurocomputing 442:98–115

    Article  Google Scholar 

  9. Jian L, Li J, Shu K, Liu H (2016) Multi-label informed feature selection. IJCAI 16:1627–33

    Google Scholar 

  10. Huang R, Jiang W, Sun G (2018) Manifold-based constraint laplacian score for multi-label feature selection. Patt Recognit Lett 112:346–352

    Article  Google Scholar 

  11. Cai Z, Zhu W (2017) Feature selection for multi-label classification using neighborhood preservation. IEEE/CAA J Automat Sinica 5(1):320–330

    Article  Google Scholar 

  12. Fan Y, Liu J, Weng W, Chen B, Chen Y, Wu S (2021) Multi-label feature selection with constraint regression and adaptive spectral graph. Knowl-Bas Sys 212:106621

    Article  Google Scholar 

  13. Zhang M-L, Zhou Z-H (2007) Ml-knn: a lazy learning approach to multi-label learning. Patt Recognit 40(7):2038–2048

    Article  MATH  Google Scholar 

  14. Zhang M-L, Zhou Z-H (2006) Multilabel neural networks with applications to functional genomics and text categorization. IEEE Trans Knowl Data Eng 18(10):1338–1351

    Article  Google Scholar 

  15. Hüllermeier E, Fürnkranz J, Cheng W, Brinker K (2008) Label ranking by learning pairwise preferences. Artif Intell 172(16–17):1897–1916

    Article  MATH  Google Scholar 

  16. Che X, Chen D, Mi J (2020) A novel approach for learning label correlation with application to feature selection of multi-label data. Infor Sci 512:795–812

    Article  MATH  Google Scholar 

  17. Wu X, Jiang B, Yu K, Chen H, Miao C (2020) Multi-label causal feature selection. In: Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 6430–6437

  18. Wang H, Chen C, Liu W, Chen K, Hu T, Chen G (2020) Incorporating label embedding and feature augmentation for multi-dimensional classification. In: Proceedings of the AAAI conference on artificial intelligence, vol. 34, pp. 6178–6185

  19. Zhu Y, Kwok JT, Zhou Z-H (2017) Multi-label learning with global and local label correlation. IEEE Trans Knowl Data Eng 30(6):1081–1094

    Article  Google Scholar 

  20. Feng L, An B, He S (2019) Collaboration based multi-label learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 33, pp. 3550–3557

  21. Dai L, Du G, Zhang J, Li C, Wei R, Li S (2020) Joint multilabel classification and feature selection based on deep canonical correlation analysis. Concurr Comput: Pract Exper 32(22):5864

    Article  Google Scholar 

  22. Du G, Zhang J, Luo Z, Ma F, Ma L, Li S (2020) Joint imbalanced classification and feature selection for hospital readmissions. Knowl-Based Sys 200:106020

    Article  Google Scholar 

  23. Lee J, Kim D-W (2015) Fast multi-label feature selection based on information-theoretic feature ranking. Patt Recognit 48(9):2761–2771

    Article  MATH  Google Scholar 

  24. Lee J, Kim D-W (2015) Memetic feature selection algorithm for multi-label classification. Infor Sci 293:80–96

    Article  Google Scholar 

  25. Zhang M-L, Peña JM, Robles V (2009) Feature selection for multi-label naive bayes classification. Infor Sci 179(19):3218–3229

    Article  MATH  Google Scholar 

  26. Hashemi A, Dowlatshahi MB, Nezamabadi-Pour H (2020) Mfs-mcdm: multi-label feature selection using multi-criteria decision making. Knowl-Based Sys 206:106365

    Article  Google Scholar 

  27. Hashemi A, Dowlatshahi MB, Nezamabadi-pour H (2021) An efficient pareto-based feature selection algorithm for multi-label classification. Infor Sci 581:428–447

    Article  Google Scholar 

  28. Sun Z, Zhang J, Dai L, Li C, Zhou C, Xin J, Li S (2019) Mutual information based multi-label feature selection via constrained convex optimization. Neurocomputing 329:447–456

    Article  Google Scholar 

  29. Nie F, Huang H, Cai X, Ding C.H.Q (2010) Efficient and robust feature selection via joint 2, 1-norms minimization. In: 24th Annual conference on neural information processing systems, Vancouver, British Columbia, Canada, pp. 1813–1821

  30. Chen S-B, Zhang Y-M, Ding CH, Zhang J, Luo B (2019) Extended adaptive lasso for multi-class and multi-label feature selection. Knowl-Based Sys 173:28–36

    Article  Google Scholar 

  31. Fan Y, Chen B, Huang W, Liu J, Weng W, Lan W (2022) Multi-label feature selection based on label correlations and feature redundancy. Knowl-Based Sys 241:108256

    Article  Google Scholar 

  32. Boyd S, Parikh N, Chu E (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Now Publishers Inc

  33. Zhang Q.-W, Zhong Y, Zhang M.-L (2018) Feature-induced labeling information enrichment for multi-label learning. In: Proceedings of the AAAI conference on artificial intelligence, vol. 32, pp. 4446–4453 (2018)

  34. Bartels RH, Stewart GW (1972) Solution of the matrix equation ax+ xb= c [f4]. Commun ACM 15(9):820–826

    Article  MATH  Google Scholar 

  35. Liu Y, Wen K, Gao Q, Gao X, Nie F (2018) Svm based multi-label learning with missing labels for image annotation. Patt Recognit 78:307–317

    Article  Google Scholar 

  36. Zhang J, Li C, Cao D, Lin Y, Su S, Dai L, Li S (2018) Multi-label learning with label-specific features by resolving label correlations. Knowl-Based Sys 159:148–157

    Article  Google Scholar 

  37. Yeh C.-K, Wu W.-C, Ko W.-J, Wang Y.-C.F (2017) Learning deep latent space for multi-label classification. In: Thirty-first AAAI conference on artificial intelligence, pp. 2838–2844

  38. Zhang J, Li C, Sun Z, Luo Z, Zhou C, Li S (2019) Towards a unified multi-source-based optimization framework for multi-label learning. Appl Soft Comput 76:425–435

    Article  Google Scholar 

  39. Lin Y, Hu Q, Liu J, Duan J (2015) Multi-label feature selection based on max-dependency and min-redundancy. Neurocomputing 168:92–103

    Article  Google Scholar 

  40. Lee J, Kim D-W (2017) Scls: multi-label feature selection based on scalable criterion for large label set. Patt Recognit 66:342–352

    Article  Google Scholar 

  41. Zhang J, Luo Z, Li C, Zhou C, Li S (2019) Manifold regularized discriminative feature selection for multi-label learning. Patt Recognit 95:136–150

    Article  Google Scholar 

  42. Demšar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7:1–30

    MATH  Google Scholar 

  43. Du G, Zhang J, Jiang M, Long J, Lin Y, Li S, Tan KC (2021) Graph-based class-imbalance learning with label enhancement. IEEE Trans Neur Netw Learn Sys. https://doi.org/10.1109/TNNLS.2021.3133262

    Article  Google Scholar 

  44. Zhang P, Liu G, Li S (2019) Distinguishing two types of labels for multi-label feature selection. Patt Recognit 95:72–82

    Article  Google Scholar 

  45. Yu K, Cai M, Li J (2021) Multilabel feature selection: a local causal structure learning approach. IIEEE Trans Neur Netw Learn Sys. https://doi.org/10.1109/TNNLS.2021.3111288

    Article  Google Scholar 

  46. Hou S, Chu Y, Fei J (2020) Intelligent global sliding mode control using recurrent feature selection neural network for active power filter. IEEE Trans Ind Electr 68(8):7320–7329

    Article  Google Scholar 

  47. Liu J, Lin Y, Du J, Zhang H, Chen Z, Zhang J (2022) ASFS: a novel streaming feature selection for multi-label data based on neighborhood rough set. Appl Intell. https://doi.org/10.1007/s10489-022-03366-x

    Article  Google Scholar 

  48. Tan A, Liang J, Wu W, Zhang J (2022) Semi-supervised partial multi-label classification via consistency learning. Patt Recognit. https://doi.org/10.1016/j.patcog.2022.108839

    Article  Google Scholar 

  49. Zhang J, Wu H, Jiang M, Liu J, Li S, Tang Y, Long J (2022) Group-preserving label-specific feature selection for multi-label learning. Exp Sys Appl. https://doi.org/10.1016/j.eswa.2022.118861

    Article  Google Scholar 

  50. Huang M, Zhuang F, Zhang X, Ao X, Niu Z, Zhang M, He Q (2019) Supervised representation learning for multi-label classification. Mach Learn 108(5):747–763

    Article  MATH  Google Scholar 

  51. Xiong Y, Wang L, Wang Q, Liu S, Kou B (2022) Improved convolutional neural network with feature selection for imbalanced ECG Multi-Factor classification. Measurement 189:110471

    Article  Google Scholar 

  52. Cui Z, Lu N (2021) Feature selection accelerated convolutional neural networks for visual tracking. Appl Intell 51(11):8230–8244

    Article  Google Scholar 

  53. CheLe T, Kim Y, Kim H (2019) Network intrusion detection based on novel feature selection model and various recurrent neural networks. Appl Sci 9(7):1392

    Article  Google Scholar 

  54. Liu X, Zhang H, Kong X, Lee K (2020) Wind speed forecasting using deep neural network with feature selection. Neurocomputing 397:393–403

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Nature Science Foundation of China (Nos. 61876159, 61806172, 62076116, 62106084 & U1705286), the China Postdoctoral Science Foundation Grant No. (2019M652257), the National Natural Science Foundation of Guangdong, China (No. 2022A1515010468), the Fundamental Research Funds for the Central Universities, Jinnan University (No. 21621026), the Science and Technology Project in Guangzhou (No. 202201010498), Guangdong Provincial Key Laboratory of Traditional Chinese Medicine Informatization (No. 2021B1212040007), Startup Foundation for Introducing Talent of Yanshan University (No. 8190550).

Author information

Author notes

  1. Liang Dai and Guodong Du have contributed equally to this work.

Authors and Affiliations

  1. Department of Artificial Intelligence, Xiamen University, Xiamen, 361005, Fujian, China

    Liang Dai & Shaozi Li

  2. College of Information Science and Technology, Jinan University, Guangzhou, 510632, Guangdong, China

    Jia Zhang

  3. School of Information Science and Engineering, Yanshan University, Qinhuangdao, 066004, Hebei, China

    Guodong Du

  4. College of Traditional Chinese Medicine, Fujian University of Traditional Chinese Medicine, Fuzhou, 350122, Fujian, China

    Candong Li

  5. Key Laboratory of Data Science and Intelligence Application, Guizhou University of Traditional Chinese Medicine, Guiyang, 550001, Guizhou, China

    Rong Wei

  6. Key Laboratory of Data Science and Intelligence Application, Minnan Normal University, Zhangzhou, 363000, Fujian, China

    Shaozi Li

Authors
  1. Liang Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jia Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guodong Du
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Candong Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rong Wei
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Shaozi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Jia Zhang or Shaozi Li.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dai, L., Zhang, J., Du, G. et al. Toward embedding-based multi-label feature selection with label and feature collaboration. Neural Comput & Applic 35, 4643–4665 (2023). https://doi.org/10.1007/s00521-022-07924-9

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-022-07924-9

Keywords

Search

Navigation