Skip to main content
SpringerLink
Log in

Unsupervised feature selection via multiple graph fusion and feature weight learning

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Unsupervised feature selection attempts to select a small number of discriminative features from original high-dimensional data and preserve the intrinsic data structure without using data labels. As an unsupervised learning task, most previous methods often use a coefficient matrix for feature reconstruction or feature projection, and a certain similarity graph is widely utilized to regularize the intrinsic structure preservation of original data in a new feature space. However, a similarity graph with poor quality could inevitably affect the final results. In addition, designing a rational and effective feature reconstruction/projection model is not easy. In this paper, we introduce a novel and effective unsupervised feature selection method via multiple graph fusion and feature weight learning (MGF2WL) to address these issues. Instead of learning the feature coefficient matrix, we directly learn the weights of different feature dimensions by introducing a feature weight matrix, and the weighted features are projected into the label space. Aiming to exploit sufficient relation of data samples, we develop a graph fusion term to fuse multiple predefined similarity graphs for learning a unified similarity graph, which is then deployed to regularize the local data structure of original data in a projected label space. Finally, we design a block coordinate descent algorithm with a convergence guarantee to solve the resulting optimization problem. Extensive experiments with sufficient analyses on various datasets are conducted to validate the efficacy of our proposed MGF2WL.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Indyk P, Motwani R. Approximate nearest neighbors: towards removing the curse of dimensionality. In: Proceedings of the 30th Annual ACM Symposium on Theory of Computing, 1998. 604–613

  2. Guyon I, Elisseeff A. An introduction to variable and feature selection. J Mach Learn Res, 2003, 3: 1157–1182

    MATH  Google Scholar 

  3. Buhlmann P, van de Geer S. Statistics for High-Dimensional Data: Methods, Theory and Applications. Berlin: Springer, 2011

    Book  MATH  Google Scholar 

  4. Shi Y, Liu J, Qi Z, et al. Learning from label proportions on high-dimensional data. Neural Networks, 2018, 103: 9–18

    Article  MATH  Google Scholar 

  5. Song S, Steinke T, Thakkar O, et al. Evading the curse of dimensionality in unconstrained private GLMs. In: Proceedings of International Conference on Artificial Intelligence and Statistics, 2021. 2638–2646

  6. Tang C, Zhu X, Liu X, et al. Learning a joint affinity graph for multiview subspace clustering. IEEE Trans Multimedia, 2019, 21: 1724–1736

    Article  Google Scholar 

  7. Tang C, Li Z, Wang J, et al. Unified one-step multi-view spectral clustering. IEEE Trans Knowl Data Eng, 2022. doi: https://doi.org/10.1109/TKDE.2022.3172687

  8. Shao L, Liu L, Li X. Feature learning for image classification via multiobjective genetic programming. IEEE Trans Neural Netw Learn Syst, 2014, 25: 1359–1371

    Article  Google Scholar 

  9. Pes B, Dessí N, Angioni M. Exploiting the ensemble paradigm for stable feature selection: a case study on high-dimensional genomic data. Inf Fusion, 2017, 35: 132–147

    Article  Google Scholar 

  10. Kasun L L C, Yang Y, Huang G B, et al. Dimension reduction with extreme learning machine. IEEE Trans Image Process, 2016, 25: 3906–3918

    Article  MathSciNet  MATH  Google Scholar 

  11. Jolliffe I T. Principal component analysis and factor analysis. In: Proceedings of Principal Component Analysis, 1986. 115–128

  12. Belhumeur P N, Hespanha J P, Kriegman D J. Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Machine Intell, 1997, 19: 711–720

    Article  Google Scholar 

  13. Liu Q, Lu H, Ma S. Improving kernel fisher discriminant analysis for face recognition. IEEE Trans Circuits Syst Video Technol, 2004, 14: 42–49

    Article  Google Scholar 

  14. Fraley C, Raftery A E. Model-based clustering, discriminant analysis, and density estimation. J Am Statistical Assoc, 2002, 97: 611–631

    Article  MathSciNet  MATH  Google Scholar 

  15. Zuo W M, Zhang D, Yang J, et al. BDPCA plus LDA: a novel fast feature extraction technique for face recognition. IEEE Trans Syst Man Cybern B, 2006, 36: 946–953

    Article  Google Scholar 

  16. Pang Y, Wang S, Yuan Y. Learning regularized LDA by clustering. IEEE Trans Neural Netw Learn Syst, 2014, 25: 2191–2201

    Article  Google Scholar 

  17. Banerjee M, Pal N R. Unsupervised feature selection with controlled redundancy (UFeSCoR). IEEE Trans Knowl Data Eng, 2015, 27: 3390–3403

    Article  Google Scholar 

  18. Mitra P, Murthy C A, Pal S K. Unsupervised feature selection using feature similarity. IEEE Trans Pattern Anal Machine Intell, 2002, 24: 301–312

    Article  Google Scholar 

  19. He X, Cai D, Niyogi P. Laplacian score for feature selection. In: Proceedings of Annual Conference on Neural Information Processing Systems, 2005. 18: 507–514

    Google Scholar 

  20. Li Z, Tang J. Unsupervised feature selection via nonnegative spectral analysis and redundancy control. IEEE Trans Image Process, 2015, 24: 5343–5355

    Article  MathSciNet  MATH  Google Scholar 

  21. Nie F, Wei Z, Li X. Unsupervised feature selection with structured graph optimization. In: Proceedings of AAAI Conference on Artificial Intelligence, 2016. 1302–1308

  22. Tang C, Zhu X, Chen J, et al. Robust graph regularized unsupervised feature selection. Expert Syst Appl, 2018, 96: 64–76

    Article  Google Scholar 

  23. Hou C, Nie F, Tao H, et al. Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Trans Knowl Data Eng, 2017, 29: 1998–2011

    Article  Google Scholar 

  24. Tang C, Liu X, Zhu X, et al. Feature selective projection with low-rank embedding and dual laplacian regularization. IEEE Trans Knowl Data Eng, 2020, 32: 1747–1760

    Google Scholar 

  25. Zhou P, Chen J, Fan M, et al. Unsupervised feature selection for balanced clustering. Knowledge-Based Syst, 2020, 193: 105417

    Article  Google Scholar 

  26. Zhang R, Zhang Y, Li X. Unsupervised feature selection via adaptive graph learning and constraint. IEEE Trans Neural Netw Learn Syst, 2022, 33: 1355–1362

    Article  MathSciNet  Google Scholar 

  27. Liao W Z, Pizurica A, Scheunders P, et al. Semisupervised local discriminant analysis for feature extraction in hyperspectral images. IEEE Trans Geosci Remote Sens, 2013, 51: 184–198

    Article  Google Scholar 

  28. Benabdeslem K, Hindawi M. Efficient semi-supervised feature selection: constraint, relevance, and redundancy. IEEE Trans Knowl Data Eng, 2014, 26: 1131–1143

    Article  Google Scholar 

  29. Zhao Z, Liu H. Spectral feature selection for supervised and unsupervised learning. In: Proceedings of International Conference on Machine Learning, 2007. 1151–1157

  30. Nie F, Huang H, Cai X, et al. Efficient and robust feature selection via joint ℓ2,1-norms minimization. In: Proceedings of Annual Conference on Neural Information Processing Systems, 2010. 1813–1821

  31. Zhou N, Xu Y, Cheng H, et al. Global and local structure preserving sparse subspace learning: an iterative approach to unsupervised feature selection. Pattern Recognition, 2016, 53: 87–101

    Article  MATH  Google Scholar 

  32. Zhu P, Zuo W, Zhang L, et al. Unsupervised feature selection by regularized self-representation. Pattern Recognition, 2015, 48: 438–446

    Article  MATH  Google Scholar 

  33. Wang S, Tang J, Liu H. Embedded unsupervised feature selection. In: Proceedings of AAAI Conference on Artificial Intelligence, 2015. 470–476

  34. Zhu L, Shen J, Xie L, et al. Unsupervised topic hypergraph hashing for efficient mobile image retrieval. IEEE Trans Cybern, 2016, 47: 3941–3954

    Article  Google Scholar 

  35. Han Y, Zhu L, Cheng Z, et al. Discrete optimal graph clustering. IEEE Trans Cybern, 2018, 50: 1697–1710

    Article  Google Scholar 

  36. Shi D, Zhu L, Li Y, et al. Robust structured graph clustering. IEEE Trans Neural Netw Learn Syst, 2019, 31: 4424–4436

    Article  MathSciNet  Google Scholar 

  37. Jiang J, Yu Y, Wang Z, et al. Graph-regularized locality-constrained joint dictionary and residual learning for face sketch synthesis. IEEE Trans Image Process, 2018, 28: 628–641

    Article  MathSciNet  MATH  Google Scholar 

  38. Zhou P, Du L, Li X, et al. Unsupervised feature selection with adaptive multiple graph learning. Pattern Recognition, 2020, 105: 107375

    Article  Google Scholar 

  39. Jiang J, Ma J, Liu X. Multilayer spectral-spatial graphs for label noisy robust hyperspectral image classification. IEEE Trans Neural Netw Learn Syst, 2022, 33: 839–852

    Article  Google Scholar 

  40. Zhang H, Wu D, Nie F, et al. Multilevel projections with adaptive neighbor graph for unsupervised multi-view feature selection. Inf Fusion, 2021, 70: 129–140

    Article  Google Scholar 

  41. Tang C, Zheng X, Liu X, et al. Cross-view locality preserved diversity and consensus learning for multi-view unsupervised feature selection. IEEE Trans Knowl Data Eng, 2022, 34: 4705–4716

    Article  Google Scholar 

  42. Zhao Z, Wang L, Liu H, et al. On similarity preserving feature selection. IEEE Trans Knowl Data Eng, 2011, 25: 619–632

    Article  Google Scholar 

  43. Zhu P, Xu Q, Hu Q, et al. Co-regularized unsupervised feature selection. Neurocomputing, 2018, 275: 2855–2863

    Article  Google Scholar 

  44. Kang Z, Shi G, Huang S, et al. Multi-graph fusion for multi-view spectral clustering. Knowledge-Based Syst, 2020, 189: 105102

    Article  Google Scholar 

  45. Gan J, Peng Z, Zhu X, et al. Brain functional connectivity analysis based on multi-graph fusion. Med Image Anal, 2021, 71: 102057

    Article  Google Scholar 

  46. Hu R, Deng Z, Zhu X. Multi-scale graph fusion for co-saliency detection. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2021. 35: 7789–7796

    Article  Google Scholar 

  47. Du L, Shen Y D. Unsupervised feature selection with adaptive structure learning. In: Proceedings of the 21st ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 2015. 209–218

  48. Nie F, Zhu W, Li X. Unsupervised feature selection with structured graph optimization. In: Proceedings of AAAI Conference on Artificial Intelligence, 2016

  49. Fan M, Chang X, Tao D. Structure regularized unsupervised discriminant feature analysis. In: Proceedings of the AAAI Conference on Artificial Intelligence, 2017

  50. Zhu P, Zhu W, Hu Q, et al. Subspace clustering guided unsupervised feature selection. Pattern Recognition, 2017, 66: 364–374

    Article  Google Scholar 

  51. Luo M, Nie F, Chang X, et al. Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst, 2017, 29: 944–956

    Article  Google Scholar 

  52. Li X, Zhang H, Zhang R, et al. Generalized uncorrelated regression with adaptive graph for unsupervised feature selection. IEEE Trans Neural Netw Learn Syst, 2018, 30: 1587–1595

    Article  MathSciNet  Google Scholar 

  53. Zhou R, Chang X, Shi L, et al. Person reidentification via multi-feature fusion with adaptive graph learning. IEEE Trans Neural Netw Learn Syst, 2019, 31: 1592–1601

    Article  Google Scholar 

  54. Li Z, Nie F, Chang X, et al. Dynamic affinity graph construction for spectral clustering using multiple features. IEEE Trans Neural Netw Learn Syst, 2018, 29: 6323–6332

    Article  MathSciNet  Google Scholar 

  55. Zhang Z, Bai L, Liang Y, et al. Joint hypergraph learning and sparse regression for feature selection. Pattern Recognition, 2017, 63: 291–309

    Article  MATH  Google Scholar 

  56. Zhu X, Zhu Y, Zhang S, et al. Adaptive hypergraph learning for unsupervised feature selection. In: Proceedings of International Joint Conference on Artificial Intelligence, 2017. 3581–3587

  57. Ding D, Yang X, Xia F, et al. Unsupervised feature selection via adaptive hypergraph regularized latent representation learning. Neurocomputing, 2020, 378: 79–97

    Article  Google Scholar 

  58. Zhong G, Pun C M. Subspace clustering by simultaneously feature selection and similarity learning. Knowledge-Based Syst, 2020, 193: 105512

    Article  Google Scholar 

  59. Yuan A, You M, He D, et al. Convex non-negative matrix factorization with adaptive graph for unsupervised feature selection. IEEE Trans Cybern, 2022, 52: 5522–5534

    Article  Google Scholar 

  60. Boyd S, Vandenberghe L. Convex Optimization. Cambridge: Cambridge University Press, 2004

    Book  MATH  Google Scholar 

  61. Li J, Cheng K, Wang S, et al. Feature selection: a data perspective. ACM Comput Surveys, 2017, 50: 1–45

    Google Scholar 

  62. Lee K-C, Ho J, Kriegman D J. Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Machine Intell, 2005, 27: 684–698

    Article  Google Scholar 

  63. Samaria F S, Harter A C. Parameterisation of a stochastic model for human face identification. In: Proceedings of IEEE Workshop on Applications of Computer Vision, 1994. 138–142

  64. Hong Z Q, Yang J Y. Optimal discriminant plane for a small number of samples and design method of classifier on the plane. Pattern Recognition, 1991, 24: 317–324

    Article  MathSciNet  Google Scholar 

  65. Du X, Yan Y, Pan P, et al. Multiple graph unsupervised feature selection. Signal Processing, 2016, 120: 754–760

    Article  Google Scholar 

  66. Zheng W, Xu C, Yang J, et al. Low-rank structure preserving for unsupervised feature selection. Neurocomputing, 2018, 314: 360–370

    Article  Google Scholar 

  67. Tang C, Liu X, Li M, et al. Robust unsupervised feature selection via dual self-representation and manifold regularization. Knowledge-Based Syst, 2018, 145: 109–120

    Article  Google Scholar 

Download references

Acknowledgements

This work was partly supported by National Key R&D Program of China (Grant No. 2020AAA0107100), National Natural Science Foundation of China (Grant No. 62076228), Natural Science Foundation of Shandong Province (Grant No. ZR2021LZH001), and Opening Fund of State Key Laboratory for Novel Software Technology, Nanjing University (Grant No. KFKT2021B24).

Author information

Authors and Affiliations

  1. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Chang Tang

  2. State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, 210023, China

    Chang Tang

  3. School of Computer, National University of Defense Technology, Changsha, 410073, China

    Xiao Zheng, Xinwang Liu & En Zhu

  4. National Supercomputing Center in Jinan, Qilu University of Technology (Shandong Academy of Sciences), Jinan, 250353, China

    Wei Zhang

  5. College of Mathematics, Physics and Information Engineering, Zhejiang Normal University, Jinhua, 321004, China

    Xinzhong Zhu

Authors
  1. Chang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xinwang Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xinzhong Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. En Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Xiao Zheng or Wei Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, C., Zheng, X., Zhang, W. et al. Unsupervised feature selection via multiple graph fusion and feature weight learning. Sci. China Inf. Sci. 66, 152101 (2023). https://doi.org/10.1007/s11432-022-3579-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-022-3579-1

Keywords

Search

Navigation