Skip to main content
Springer Nature Link
Log in

Adaptive and flexible \(\ell _1\)-norm graph embedding for unsupervised feature selection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Unsupervised feature selection (UFS) is a fundamental and indispensable dimension reduction method for large amount of high-dimensional unlabeled data samples. Without label information, the manifold learning technique is leveraged to compensate for the lack of discrimination with the selected features. However, it is still a challenging problem to capture the geometrical structure for practical data, which are often contaminated by noises and outliers. Additionally, the predetermined graph embedded UFS models suffer from the parameter tuning problem and the separated model optimization procedures. To generate more compact and discriminative feature subsets, we propose a Robust UFS model with Adaptive and Flexible \(\varvec{\ell }_\textbf{1}\)-norm Graph (RAFG) embedding. Specifically, the \(\varvec{\ell }_\textbf{2,1}\)-norm is imposed on the flexible regression term to alleviate the adverse effects of both noisy features and outliers, and \(\varvec{\ell }_\textbf{2,p}\)-norm regularization term is incorporated to ensure that the selected transformation matrix is sufficiently sparse. Moreover, the adaptive \(\varvec{\ell }_\textbf{1}\)-norm graph learning characterize the clustering distribution via consistent embeddings, which avoids time-consuming distance computations in a high-dimensional feature space. To solve the challenging problem, we propose an efficient alternative updating algorithm with an iterative reweighted strategy, together with the necessary convergence and complexity analyses. Finally, experimental results on two synthetic data and eight benchmark datasets illustrate the effectiveness and superiority of the proposed RAFG method compared with state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Algorithm 1
Algorithm 2
Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Data Availability

All data generated or analysed during this study are publicly available or included in this published article.

References

  1. Yan S, Xu D, Zhang B, H.j. Zhang, Q. Yang, S. Lin (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51. https://doi.org/10.1109/TPAMI.2007.250598

  2. He X, Cai D, Niyogi P (2005) Laplacian Score for Feature Selection. In: Proceedings of the 18th international conference on neural information processing systems (MIT Press, Cambridge, MA, USA), NIPS’05, pp 507–514

  3. Lai Z, Mo D, Wong WK, Xu Y, Miao D, Zhang D (2018) Robust discriminant regression for feature extraction. IEEE Transactions on Cybernetics 48(8):2472–2484. https://doi.org/10.1109/TCYB.2017.2740949

    Article  Google Scholar 

  4. Gui J, Sun Z, Ji S, Tao D, Tan T (2017) Feature selection based on structured sparsity: a comprehensive study. IEEE Transactions on Neural Networks and Learning Systems 28(7):1490–1507. https://doi.org/10.1109/TNNLS.2016.2551724

    Article  MathSciNet  Google Scholar 

  5. Zhao YP, Chen L, Chen CLP (2021) Laplacian regularized nonnegative representation for clustering and dimensionality reduction. IEEE Trans Circuits Syst Video Technol 31(1):1–14. https://doi.org/10.1109/TCSVT.2020.2967424

    Article  MathSciNet  Google Scholar 

  6. Li X, Zhang H, Zhang R, Liu Y, Nie F (2019) Generalized uncorrelated regression with adaptive graph for unsupervised feature selection. IEEE Transactions on Neural Networks and Learning Systems 30(5):1587–1595. https://doi.org/10.1109/TNNLS.2018.2868847

    Article  MathSciNet  Google Scholar 

  7. Nie F, Wang H, Huang H, Ding C (2011) Unsupervised and semi-supervised learning via \(\ell _1\)-norm graph. In: 2011 International conference on computer vision, pp 2268–2273. https://doi.org/10.1109/ICCV.2011.6126506

  8. Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Proceedings of the thirtieth AAAI conference on artificial intelligence (AAAI Press), AAAI’16, pp 1302–1308

  9. Sun J, Wang Z, Wang W, Li H, Sun F, Ding Z (2022) Joint adaptive dual graph and feature selection for domain adaptation. IEEE Trans Circuits Syst Video Technol 32(3):1453–1466. https://doi.org/10.1109/TCSVT.2021.3073937

    Article  Google Scholar 

  10. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining (Association for Computing Machinery, New York, USA), KDD ’10, pp 333–342. https://doi.org/10.1145/1835804.1835848

  11. Du L, Shen YD (2015) Unsupervised feature selection with adaptive structure learning. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining (Association for Computing Machinery, New York, USA), KDD ’15, pp 209–218

  12. Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2018) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Transactions on Cybernetics 48(2):648–660. https://doi.org/10.1109/TCYB.2017.2647904

    Article  Google Scholar 

  13. Du S, Ma Y, Li S, Ma Y (2017) Robust unsupervised feature selection via matrix factorization. Neurocomputing 241:115–127. https://doi.org/10.1016/j.neucom.2017.02.034

  14. Qian M, Zhai C (2013) Robust unsupervised feature selection. In: Proceedings of the twenty-third international joint conference on artificial intelligence (AAAI Press), IJCAI ’13, pp 1621–1627

  15. Wang S, Tang J, Liu H (2015) Embedded unsupervised feature selection. In: Proceedings of the twenty-ninth AAAI conference on artificial intelligence (AAAI Press), AAAI’15, pp 470–476

  16. Zhu P, Hu Q, Zhang C, Zuo W (2016) Coupled dictionary learning for unsupervised feature selection. In: Proceedings of the thirtieth AAAI conference on artificial intelligence (AAAI Press), AAAI’16, pp 2422–2428

  17. Miao J, Yang T, Fan C, Chen Z, Fei X, Ju X, Wang K, Xu M (2022) Self-paced non-convex regularized analysis-synthesis dictionary learning for unsupervised feature selection. Knowl-Based Systs 241:108279. https://doi.org/10.1016/j.knosys.2022.108279

  18. Zhao Z, Wang L, Liu H (2010) Efficient spectral feature selection with minimum redundancy. In: Proceedings of the twenty-fourth AAAI conference on artificial intelligence (AAAI Press), AAAI’10, pp 673–678

  19. Li Z, Yang Y, Liu J, Zhou X, Lu H (2014) Unsupervised feature selection using nonnegative spectral analysis. In: Proceedings of the twenty-sixth AAAI conference on artificial intelligence (AAAI Press, 2012), AAAI’12, pp 1026–1032

  20. Shi L, Du L, Shen YD (2014) Robust spectral learning for unsupervised feature selection. In: 2014 IEEE International conference on data mining, pp 977–982. https://doi.org/10.1109/ICDM.2014.58

  21. Hou C, Nie F, Yi D, Wu Y (2011) Feature selection via joint embedding learning and sparse regression. In: Proceedings of the twenty-second international joint conference on artificial intelligence - volume volume two (AAAI Press), IJCAI’11, pp 1324–1329

  22. Wang S, Pedrycz W, Zhu Q, Zhu W (2015) Subspace learning for unsupervised feature selection via matrix factorization. Pattern Recognition 48(1):10–19. https://doi.org/10.1016/j.patcog.2014.08.004

  23. Zhu P, Zhu W, Hu Q, Zhang C, Zuo W (2017 Subspace clustering guided unsupervised feature selection. Pattern Recog 66:364–374). https://doi.org/10.1016/j.patcog.2017.01.016

  24. Li J, Tang J, Liu H (2017) Reconstruction-based unsupervised feature selection: an embedded approach. In: Proceedings of the twenty-sixth international joint conference on artificial intelligence, IJCAI-17, pp 2159–2165. https://doi.org/10.24963/ijcai.2017/300

  25. Liu Y, Ye D, Li W, Wang H, Gao Y (2020) Robust neighborhood embedding for unsupervised feature selection. Knowl-Based Systs 193:105462. https://doi.org/10.1016/j.knosys.2019.105462

    Article  Google Scholar 

  26. Shi D, Zhu L, Li Y, Li J, Nie X (2020) Robust structured graph clustering. IEEE Transactions on Neural Networks and Learning Systems 31(11):4424–4436. https://doi.org/10.1109/TNNLS.2019.2955209

    Article  MathSciNet  Google Scholar 

  27. Nie F, Zhu W, Li X (2021) Structured graph optimization for unsupervised feature selection. IEEE Transactions on Knowledge and Data Engineering 33(3):1210–1222 ). https://doi.org/10.1109/TKDE.2019.2937924

  28. Tang C, Zhu X, Chen J, Wang P, Liu X, Tian J (2018) Robust graph regularized unsupervised feature selection. Expert Syst Appl 96:64–76. https://doi.org/10.1016/j.eswa.2017.11.053

    Article  Google Scholar 

  29. Miao J, Yang T, Sun L, Fei X, Niu L, Shi Y (2022) Graph regularized locally linear embedding for unsupervised feature selection. Pattern Recog 122:108299. https://doi.org/10.1016/j.patcog.2021.108299

  30. Nie F, Wang H, Huang H, Ding C (2013) Adaptive loss minimization for semi-supervised elastic embedding. In: Proceedings of the twenty-third international joint conference on artificial intelligence (AAAI Press), IJCAI ’13, pp 1565–1571

  31. Luo M, Chang X, Nie L, Yang Y, Hauptmann AG, Zheng Q (2018) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Transactions on Cybernetics 48(2):648–660. https://doi.org/10.1109/TCYB.2017.2647904

    Article  Google Scholar 

  32. Chen H, Nie F, Wang R, Li X (2022) Unsupervised feature selection with flexible optimal graph. IEEE Transactions on neural networks and learning systems pp 1–14. https://doi.org/10.1109/TNNLS.2022.3186171

  33. Yu YF, Xu G, Huang KK, Zhu H, Chen L, Wang H (2021) Dual calibration mechanism based l2, p-norm for graph matching. IEEE Trans Circuits Syst Video Technol 31(6):2343–2358. https://doi.org/10.1109/TCSVT.2020.3023781

    Article  Google Scholar 

  34. Liu X, Wang L, Zhang J, Yin J, Liu H (2014) Global and local structure preservation for feature selection. IEEE Transactions on Neural Networks and Learning Systems 25(6):1083–1095. https://doi.org/10.1109/TNNLS.2013.2287275

    Article  Google Scholar 

  35. Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416. https://doi.org/10.1007/s11222-007-9033-z

  36. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905. https://doi.org/10.1109/34.868688

    Article  Google Scholar 

  37. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326. https://doi.org/10.1126/science.290.5500.2323

    Article  Google Scholar 

  38. Ma Z, Wang J, Li H, Huang Y (2023) Adaptive graph regularized non-negative matrix factorization with self-weighted learning for data clustering. Appl Intell 53(23):28054–28073. https://doi.org/10.1007/s10489-023-04868-y

    Article  Google Scholar 

  39. Xie X, Cao Z, Sun F (2023) Joint learning of graph and latent representation for unsupervised feature selection. Appl Intell 53(21):25282–25295. https://doi.org/10.1007/s10489-023-04893-x

    Article  Google Scholar 

  40. Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) L2,1-Norm regularized discriminative feature selection for unsupervised learning. In: Proceedings of the twenty-second international joint conference on artificial intelligence - volume volume two (AAAI Press), IJCAI’11, pp 1589–1594

  41. Zhou N, Xu Y, Cheng H, Fang J, Pedrycz W (2016) Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection. Pattern Recog pp 53:87–101. https://doi.org/10.1016/j.patcog.2015.12.008

  42. Guo J, Guo Y, Kong X, He R (2017) Unsupervised feature selection with ordinal locality. In: 2017 IEEE International conference on multimedia and expo (ICME), pp 1213–1218. https://doi.org/10.1109/ICME.2017.8019357

  43. Zhu X, Zhang S, Hu R, Zhu Y, Song J (2018) Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Transactions on knowledge and data engineering 30(3):517–529. https://doi.org/10.1109/TKDE.2017.2763618

  44. Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining (Association for Computing Machinery, New York, USA), KDD ’14, pp 977–986. https://doi.org/10.1145/2623330.2623726

  45. Yang S, Zhang R, Nie F, Li X (2019) Unsupervised feature selection based on reconstruction error minimization. In: ICASSP 2019 - 2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 2107–2111. https://doi.org/10.1109/ICASSP.2019.8682731

  46. Li W, Chen H, Li T, Wan J, Sang B (2022) Unsupervised feature selection via self-paced learning and low-redundant regularization. Knowl-Based Systs 240:108150. https://doi.org/10.1016/j.knosys.2022.108150

  47. Shang R, Zhang W, Lu M, Jiao L, Li Y (2022) Feature selection based on non-negative spectral feature learning and adaptive rank constraint. Knowl-Based Systs 236:107749

    Article  Google Scholar 

  48. Zhu P, Zuo W, Zhang L, Hu Q, Shiu SC (2015) Unsupervised feature selection by regularized self-representation. Pattern Recog 48(2):438–446. https://doi.org/10.1016/j.patcog.2014.08.006

  49. Wu JS, Liu JX, Wu JY, Huang W (2023) Dictionary learning for unsupervised feature selection via dual sparse regression. Appl Intell 53(15):18840–18856. https://doi.org/10.1007/s10489-023-04480-0

    Article  Google Scholar 

  50. Wu JS, Song MX, Min W, Lai JH, Zheng WS (2021) Joint adaptive manifold and embedding learning for unsupervised feature selection. Pattern Recog 112:107742. https://doi.org/10.1016/j.patcog.2020.107742

  51. Ling Y, Nie F, Yu W, Li X (2023) Discriminative and robust autoencoders for unsupervised feature selection. IEEE Transactions on neural networks and learning systems early access, pp 1–15. https://doi.org/10.1109/TNNLS.2023.3333737

  52. Wang S, Ding Z, Fu Y (2017) Feature selection guided auto-encoder. In: Proceedings of the thirty-first AAAI conference on artificial intelligence (AAAI Press), AAAI’17, pp 2725–2731

  53. Han K, Wang Y, Zhang C, Li C, Xu C (2018) Autoencoder inspired unsupervised feature selection. In: 2018 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 2941–2945. https://doi.org/10.1109/ICASSP.2018.8462261

  54. Qiu Z, Zeng W, Liao D, Gui N (2022) A-sfs: semi-supervised feature selection based on multi-task self-supervision. Knowl-Based Systs 252:109449. https://doi.org/10.1016/j.knosys.2022.109449

  55. Tan J, Gui N, Qiu Z (2024) Gaefs: self-supervised graph auto-encoder enhanced feature selection. Knowl-Based Syst 290:111523. https://doi.org/10.1016/j.knosys.2024.111523. https://www.sciencedirect.com/science/article/pii/S0950705124001588

  56. Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932. https://doi.org/10.1109/TIP.2010.2044958

    Article  MathSciNet  Google Scholar 

  57. Huang J, Nie F, Huang H (2015) A new simplex sparse learning model to measure data similarity for clustering. In: Proceedings of the 24th international conference on artificial intelligence (AAAI Press), IJCAI’15, pp 3569–3575

  58. Nie F, Huang H, Cai X, Ding C (2010) Efficient and robust feature selection via joint \(\ell _{2,1}\)-norms minimization. In: Proceedings of the 23rd international conference on neural information processing systems - vol 2 (Curran Associates Inc., Red Hook, USA), NIPS’10, pp 1813–1821

  59. Tang C, Liu X, Li M, Wang P, Chen J, Wang L, Li W (2018) Robust unsupervised feature selection via dual self-representation and manifold regularization. Knowl-Based Systs 145:109–120. https://doi.org/10.1016/j.knosys.2018.01.009

  60. Wu Y, Wang Y, Li Y, Zhu X, Wu X (2022) Top-k self-adaptive contrast sequential pattern mining. IEEE Transactions on Cybernetics 52(11):11819–11833. https://doi.org/10.1109/TCYB.2021.3082114

    Article  Google Scholar 

Download references

Funding

This work is supported by the Scientific Research Program Funded by Education Department of Shaanxi Provincial Government (No. 23JP107), Shaanxi Province Key Research and Development Program (No. 2022ZDLSF07-07), Open Project of Key Laboratory of Road Construction Technology and Equipment Ministry of Education (Chang’an University) (No. 300102252510) and Special Project of Technological Innovation and Guidance in Shaanxi Province (No. 2022QFY01-03).

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Xi’an University of Technology, Xi’an, China

    Kun Jiang, Ting Cao, Lei Zhu & Qindong Sun

  2. School of Cyber Science and Engineering, Xi’an Jiaotong University, Xi’an, China

    Qindong Sun

Authors
  1. Kun Jiang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ting Cao
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Lei Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Qindong Sun
    View author publications

    You can also search for this author inPubMed Google Scholar

Contributions

Kun Jiang contributed to the conception, design and experiment of the study, and wrote the manuscript; Ting Cao helped revise the manuscript concerning the language, grammar issues; Lei Zhu and Qindong Sun helped perform the experimental analysis with constructive discussions.

Corresponding author

Correspondence to Ting Cao.

Ethics declarations

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding the publication of this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jiang, K., Cao, T., Zhu, L. et al. Adaptive and flexible \(\ell _1\)-norm graph embedding for unsupervised feature selection. Appl Intell 54, 11732–11751 (2024). https://doi.org/10.1007/s10489-024-05760-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-024-05760-z

Keywords