Skip to main content
SpringerLink
Log in

Multi-label feature selection based on logistic regression and manifold learning

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Like traditional single-label learning, multi-label learning is also faced with the problem of dimensional disaster. Feature selection is an effective technique for dimensionality reduction and learning efficiency improvement of high-dimensional data. This paper combined logistic regression, manifold learning, and sparse regularization to construct a joint framework for multi-label feature selection (LMFS). Firstly, the sparsity of the feature weight matrix is constrained by the L2, 1-norm. Secondly, the feature manifold and label manifold can constrain the feature weight matrix to fit the information of data and label better. An iterative updating algorithm is designed, and the convergence of the algorithm is proved. Finally, the LMFS algorithm is compared with DRMFS, SCLS, and other algorithms on eight classical multi-label data sets. The experimental results show the effectiveness of the LMFS algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Cai J, Luo JW, Wang SL et al (2018) Feature selection in machine learning: a new perspective[J]. Neurocomputing 300:70–79

    Article  Google Scholar 

  2. Tang C, Liu XW, Zhu XZ et al (2019) Feature selective projection with low-rank embedding and dual laplacian regularization[J]. IEEE Trans Knowl Data Eng 32(9):1747–1760

    Google Scholar 

  3. Bermingham ML, Pong-Wong R, Spiliopoulou A et al (2015) Application of high-dimensional feature selection: evaluation for genomic prediction in man[J]. Scientific Reports 5:10312

    Article  Google Scholar 

  4. Sun X, Liu Y H, Li J et al (2012) Using cooperative game theory to optimize the feature selection problem[J]. Neurocomputing 97:86–93

    Article  Google Scholar 

  5. Zhang R, Nie FP, Li XL et al (2019) Feature selection with multi-view data: a survey[J]. Information Fusion 50:158–167

    Article  Google Scholar 

  6. Ding CC, Zhao M, Lin J et al (2019) Multi-objective iterative optimization algorithm based optimal wavelet filter selection for multi-fault diagnosis of rolling element bearings[J]. ISA Transactions 82:199–215

    Article  Google Scholar 

  7. Labani M, Moradi P, Ahmadizar F et al (2018) A novel multivariate filter method for feature selection in text classification problems[J]. Engineering Applications of Artificial Intelligence 70:25–37

    Article  Google Scholar 

  8. Yao C, Liu YF, Jiang B et al (2017) LLE score: a new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition[J]. IEEE Transactions on Image Processing 26(11):5257–5269

    Article  MathSciNet  Google Scholar 

  9. Gonzalez J, Ortega J, Damas M et al (2019) A new multi-objective wrapper method for feature selection - accuracy and stability analysis for BCI[J]. Neurocomputing 333:407–418

    Article  Google Scholar 

  10. Swati J, Hongmei H, Karl J (2018) Information gain directed genetic algorithm wrapper feature selection for credit rating[J]. Applied Soft Computing 69:541–553

    Article  Google Scholar 

  11. Maldonado S, López J (2018) Dealing with high-dimensional class-imbalanced datasets: embedded feature selection for SVM classification[J]. Applied Soft Computing 67:94–105

    Article  Google Scholar 

  12. Kong YC, Yu TW (2018) A graph-embedded deep feedforward network for disease outcome classification and feature selection using gene expression data[J]. Bioinformatics 34(21):3727– 3737

    Article  Google Scholar 

  13. Yang XH, Jiang XY, Tian CX et al (2020) Inverse projection group sparse representation for tumor classification: a low rank variation dictionary approach[J]. Knowledge-Based Systems 196:105768

    Article  Google Scholar 

  14. Deng T Q, Ye D S, Ma R, et al. (2020) Low-rank local tangent space embedding for subspace clustering[J]. Inf Sci 508:1–21

    Article  MathSciNet  Google Scholar 

  15. Xiao Q, Dai JH, Luo JW et al (2019) Multi-view manifold regularized learning-based method for prioritizing candidate disease miRNAs[J]. Knowledge-Based Systems 175:118–129

    Article  Google Scholar 

  16. Tang C, Zheng X, Liu XW et al (2021) Cross-view locality preserved diversity and consensus learning for multi-view unsupervised feature selection[J]. IEEE Transactions on Knowledge and Data Engineering 99:1–1

    Google Scholar 

  17. Tang C, Liu XW, Li MM et al (2018) Robust unsupervised feature selection via dual self-representation and manifold regularization[J]. Knowledge-Based Systems 145:109–120

    Article  Google Scholar 

  18. Sun ZQ, Zhang J, Dai L et al (2019) Mutual information based multi-label feature selection via constrained convex optimization[J]. Neurocomputing 329:447–456

    Article  Google Scholar 

  19. Zhang P, Liu GX, Gao WF (2019) Distinguishing two types of labels for multi-label feature selection[J]. Pattern Recogn 95:72–82

    Article  Google Scholar 

  20. Chen LL, Chen DG (2019) Alignment based feature selection for multi-label learning[J]. Neural Processing Letters 50(7):28–36

    Google Scholar 

  21. Chen SB, Zhang YM, Ding CHQ et al (2019) Extended adaptive lasso for multi-class and multi-label feature selection[J]. Knowledge-Based Systems 173:28–36

    Article  Google Scholar 

  22. Zhang J, Luo ZM, Li CD et al (2019) Manifold regularized discriminative feature selection for multi-label learning[J]. Pattern Recognition 95:136–150

    Article  Google Scholar 

  23. Cai ZL, Zhu W (2018) Multi-label feature selection via feature manifold learning and sparsity regularization[J]. Int J Machine Learning Cybern 9(8):1321–1334

    Article  Google Scholar 

  24. Hu JC, Li YH, Gao WF et al (2020) Robust multi-label feature selection with dual-graph regularization[J]. Knowledge-Based Systems 203:106126

    Article  Google Scholar 

  25. Li Q, Xie B, You J et al (2016) Correlated logistic model with elastic net regularization for multilabel image classification[J]. IEEE Transactions on Image Processing 25(8):3801–3813

    Article  MathSciNet  Google Scholar 

  26. Sato T, Takano Y, Miyashiro R et al (2016) Feature subset selection for logistic regression via mixed integer optimization[J]. Computational Optimization and Applications 64(3):865– 880

    Article  MathSciNet  Google Scholar 

  27. Yang ZY, Liang Y, Zhang H et al (2018) Robust sparse logistic regression with the Lq (0 < q < 1) regularization for feature selection using gene expression data[J]. IEEE Access 6:68586–68595

    Article  Google Scholar 

  28. Pan XL, Xu YT (2021) A safe feature elimination rule for L1-regularized logistic regression[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2021.3071138

  29. Zhang R, Nie FP, Li X (2017) Self-weighted supervised discriminative feature selection[J]. IEEE Trans Neural Netw Learn Syst 29(8):3913–3918

    Google Scholar 

  30. Zhang ML, Zhou ZH (2007) ML-KNN: a lazy learning approach to multi-label learning[J]. Pattern Recognition 40(7):2038–2048

    Article  Google Scholar 

  31. Lee J, Kim DW (2017) SCLS: multi-label feature selection based on scalable criterion for large label set[J]. Pattern Recognition 66:342–352

    Article  MathSciNet  Google Scholar 

  32. Lin YJ, Hu QH, Liu JH et al (2015) Multi-label feature selection based on max-dependency and min-redundancy[J]. Neurocomputing 168:92–103

    Article  Google Scholar 

  33. Lee J, Lim H, Kim DW (2012) Approximating mutual information for multi-label feature selection[J]. Electron Lett 48(15):929– 930

    Article  Google Scholar 

  34. Lee J, Kim DW (2015) Fast multi-label feature selection based on information-theoretic feature ranking[J]. Pattern Recognition 48(9):2761–2771

    Article  Google Scholar 

  35. Dougherty J, Kohavi R, Sahami M et al (1995) Supervised and unsupervised discretization of continuous features[J]. In: Machine learning: proceedings of the 12th international conference, vol 2, pp 194–202

  36. Demiar J, Schuurmans D (2006) Statistical comparisons of classifiers over multiple data sets[J]. Journal of Machine Learning Research 7(1):1–30

    MathSciNet  Google Scholar 

Download references

Acknowledgments

This work was supported by the Natural Science Foundation of China (61976130), the Key Research and Development Project of Shaanxi Province (2018KW-021), the Natural Science Foundation of Shaanxi Province (2020JQ-923).

Author information

Authors and Affiliations

  1. School of Science, Xi’an Polytechnic University, Xi’an, China

    Yao Zhang, Yingcang Ma & Xiaofei Yang

  2. Basic Education Department, Shandong Huayu University of Technology, Dezhou, China

    Yao Zhang

Authors
  1. Yao Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yingcang Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaofei Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yingcang Ma.

Ethics declarations

Conflict of Interests

The authors declared that they have no conflicts of interest to this work. We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Ma, Y. & Yang, X. Multi-label feature selection based on logistic regression and manifold learning. Appl Intell 52, 9256–9273 (2022). https://doi.org/10.1007/s10489-021-03008-8

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-03008-8

Keywords

Search

Navigation