Skip to main content
SpringerLink
Log in

Joint dictionary and graph learning for unsupervised feature selection

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

With the explosion of unlabelled and high-dimensional data, unsupervised feature selection has become an critical and challenging problem in machine learning. Recently, data representation based model has been successfully deployed for unsupervised feature selection, which defines feature importance as the capability to represent original data via a reconstruction function. However, most existing algorithms conduct feature selection on original feature space, which will be affected by the noisy and redundant features of original feature space. In this paper, we investigate how to conduct feature selection on the dictionary basis space of the data, which can capture higher level and more abstract representation than original low-level representation. In addition, a similarity graph is learned simultaneously to preserve the local geometrical data structure which has been confirmed critical for unsupervised feature selection. In summary, we propose a model (referred to as DGL-UFS briefly) to integrate dictionary learning, similarity graph learning and feature selection into a uniform framework. Experiments on various types of real world datasets demonstrate the effectiveness of the proposed framework DGL-UFS.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

Notes

  1. http://featureselection.asu.edu/datasets.php

References

  1. Bartels RH, Stewart GW (1972) Solution of the matrix equation ax + xb = c. Commun ACM 15(9):820–826

    Article  MATH  Google Scholar 

  2. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J (2010) Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations & Trends in Machine Learning 3(1):1–122

    Article  MATH  Google Scholar 

  3. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: ACM SIGKDD conference on knowledge discovery and data mining, pp 333–342

  4. Cai W (2017) A dimension reduction algorithm preserving both global and local clustering structure. Knowl-Based Syst 118:191–203

    Article  Google Scholar 

  5. Chang T, Jiajia C, Xinwang L, Miaomiao L, Pichao W, Minhui W, Peng L (2018) Consensus learning guided multi-view unsupervised feature selection. Knowl-Based Syst 160:49–60

    Article  Google Scholar 

  6. Dietterich TG, Bakiri G (1991) A general method for improving multi-class inductive learning programs. In: AAAI conference on artificial intelligence, pp 572–577

  7. Dy JG, Brodley CE, Kak A, Broderick LS, Aisen AM (2003) Unsupervised feature selection applied to content-based retrieval of lung images. IEEE Trans Pattern Anal Mach Intell 25(3):373–378

    Article  Google Scholar 

  8. Golay J, Kanevski M (2017) Unsupervised feature selection based on the morisita estimator of intrinsic dimension. Knowl-Based Syst 135:125–134

    Article  Google Scholar 

  9. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  10. Hall MA (2000) Correlation-based feature selection for discrete and numeric class machine learning. In: International conference on machine learning, pp 359–366

  11. He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: Advances in neural information processing systems, pp 507–514

  12. Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Transactions on Cybernetics 44(6):793–804

    Article  Google Scholar 

  13. Hu H, Rong W, Nie F, Yang X, Yu W (2018) Fast unsupervised feature selection with anchor graph and l2,1 -norm regularization. Multimed Tools Appl 77(17):22099–22113

    Article  Google Scholar 

  14. Jain A, Zongker D (1997) Feature selection: evaluation, application, and small sample performance. IEEE Trans Pattern Anal Mach Intell 19(2):153–158

    Article  Google Scholar 

  15. Javed S, Sobral A, Bouwmans T, Jung SK (2015) Or-pca with dynamic feature selection for robust background subtraction. In: The ACM/SIGAPP symposium on applied computing

  16. Khan J, Wei JS, Ringner M, Saal LH, Ladanyi M, Westermann F, Berthold F, Schwab M, Antonescu CR, Peterson C et al (2001) Classification and diagnostic prediction of cancers using gene expression profiling and artificial neural networks. Nat Med 7(6):673–679

    Article  Google Scholar 

  17. Kohavi R, John GH (1997) Wrappers for feature subset selection. Artif Intell 97(1-2):273–324

    Article  MATH  Google Scholar 

  18. Li Z, Yang Y, Liu J, Zhou X, Lu H (2012) Unsupervised feature selection using nonnegative spectral analysis. In: AAAI conference on artificial intelligence, pp 1026–1032

  19. Liao W, Pizurica A, Scheunders P, Philips W, Pi Y (2013) Semisupervised local discriminant analysis for feature extraction in hyperspectral images. IEEE Trans Geosci Remote Sens 51(1):184–198

    Article  Google Scholar 

  20. Liu X, Wang L, Zhang J, Yin J, Liu H (2014) Global and local structure preservation for feature selection. IEEE Transactions on Neural Networks and Learning Systems 25(6):1083–1095

    Article  Google Scholar 

  21. Lu G, Bo L, Yang W, Jian Y (2017) Unsupervised feature selection with graph learning via low-rank constraint. Multimed Tools Appl 77(22):29531–29549

    Article  Google Scholar 

  22. Luo M, Nie F, Chang X, Yang Y, Hauptmann AG, Zheng Q (2017) Adaptive unsupervised feature selection with structure regularization. IEEE Transactions on Neural Networks and Learning Systems 29(4):944–956

    Article  Google Scholar 

  23. Mairal J, Bach F, Ponce J, Sapiro G (2010) Online learning for matrix factorization and sparse coding. J Mach Learn Res 11(Jan):19–60

    MathSciNet  MATH  Google Scholar 

  24. Mitra P, Murthy C, Pal SK (2002) Unsupervised feature selection using feature similarity. IEEE Trans Pattern Anal Mach Intell 24(3):301–312

    Article  Google Scholar 

  25. Nie F, Wei Z, Li X (2016) Unsupervised feature selection with structured graph optimization. In: AAAI conference on artificial intelligence, pp 1302–1308

  26. Passalis N, Tefas A (2018) Pysef: a python library for similarity-based dimensionality reduction. Knowl-Based Syst 152:186–187

    Article  Google Scholar 

  27. Rate C, Retrieval C (2011) Columbia object image library (coil-20). Computer

  28. Shang R, Zhang Z, Jiao L, Liu C, Li Y (2016) Self-representation based dual-graph regularized feature selection clustering. Neurocomputing 171(1):1242–1253

    Article  Google Scholar 

  29. Singh D, Febbo PG, Ross K, Jackson DG, Manola J, Ladd C, Tamayo P, Renshaw AA, D’Amico AV, Richie JP et al (2002) Gene expression correlates of clinical prostate cancer behavior. Cancer cell 1(2):203–209

    Article  Google Scholar 

  30. Tabakhi S, Moradi P, Akhlaghian F (2014) An unsupervised feature selection algorithm based on ant colony optimization. Eng Appl Artif Intell 32(6):112–123

    Article  Google Scholar 

  31. Tang C, Cao L, Zheng X, Wang M (2018) Gene selection for microarray data classification via subspace learning and manifold regularization. Med Biol Eng Comput 56(7):1271–1284

    Article  Google Scholar 

  32. Tang C, Li W, Wang P, Wang L (2018) Online human action recognition based on incremental learning of weighted covariance descriptors. Inf Sci 467:219–237

    Article  Google Scholar 

  33. Tang C, Liu X, Li M, Wang P, Chen J, Wang L, Li W (2018) Robust unsupervised feature selection via dual self-representation and manifold regularization. Knowl-Based Syst 145:109–120

    Article  Google Scholar 

  34. Tang C, Liu X, Wang P, Zhang C, Li M, Wang L (2019) Adaptive hypergraph embedded semi-supervised multi-label image annotation. IEEE Trans Multimedia, pp 1–1. https://doi.org/10.1109/TMM.2019.2909860

  35. Tang C, Liu X, Zhu X, Xiong J, Li M, Xia J, Wang X, Wang L (2019) Feature selective projection with low-rank embedding and dual laplacian regularization. IEEE Trans Knowl Data Eng, pp 1–1. https://doi.org/10.1109/TKDE.2019.2911946

  36. Tang C, Zhu X, Liu X, Li M, Wang P, Zhang C, Wang L (2019) Learning a joint affinity graph for multiview subspace clustering. IEEE Trans Multimedia 21(7):1724–1736. https://doi.org/10.1109/TMM.2018.2889560

    Article  Google Scholar 

  37. Tang C, Zhu X, Liu X, Wang L (2019) Cross-view local structure preserved diversity and consensus learning for multi-view unsupervised feature selection. In: AAAI conference on artificial intelligence

  38. Tang C, Zhu X, Liu X, Wang L, Albert Z (2019) Defusionnet: Defocus blur detection via recurrently fusing and refining multi-scale deep features. In: IEEE computer society conference on computer vision and pattern recognition, pp 2700–2709

  39. Tang J, Alelyani S, Liu H (2014) Feature selection for classification: a review. Data Classification: Algorithms and Applications. CRC Press, 37–64

  40. Wang S, Pedrycz W, Zhu Q, Zhu W (2015) Subspace learning for unsupervised feature selection via matrix factorization. Pattern Recogn 48(1):10–19

    Article  MATH  Google Scholar 

  41. Wang S, Tang J, Liu H (2015) Embedded unsupervised feature selection. In: AAAI conference on artificial intelligence, pp 470–476

  42. Wang S, Wang H (2017) Unsupervised feature selection via low-rank approximation and structure learning. Knowl-Based Syst 124:70–79

    Article  Google Scholar 

  43. Wei H, Zhu X, Cheng D, Hu R, Zhang S (2017) Low-rank unsupervised graph feature selection via feature self-representation. Multimed Tools Appl 76(9):12149–12164

    Article  Google Scholar 

  44. Wolf L, Shashua A (2003) Feature selection for unsupervised and supervised inference: the emergence of sparsity in a weighted-based approach. In: IEEE international conference on computer vision, pp 378–384

  45. Yang XK, He L, Qu D, Zhang WQ (2016) Semi-supervised minimum redundancy maximum relevance feature selection for audio classification. Multimed Tools Appl 77(1):1–27

    Article  Google Scholar 

  46. Yang Y, Ma Z, Hauptmann AG, Sebe N (2013) Feature selection for multimedia analysis by sharing information among multiple tasks. IEEE Trans Multimedia 15(3):661–669

    Article  Google Scholar 

  47. Yang Z, Wang H, Han Y, Zhu X (2017) Discriminative multi-task multi-view feature selection and fusion for multimedia analysis. Multimed Tools Appl 77(3):3431–3453

    Article  Google Scholar 

  48. Yu G, Zhang G, Zhang Z, Yu Z, Deng L (2015) Semi-supervised classification based on subspace sparse representation. Knowl Inf Syst 43(1):81–101

    Article  Google Scholar 

  49. Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: International conference on machine learning, pp 1151–1157

  50. Zhao Z, Wang L, Liu H et al (2010) Efficient spectral feature selection with minimum redundancy. In: AAAI conference on artificial intelligence

  51. Zheng W, Zhu X, Zhu Y, Hu R, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimed Tools Appl 77(22):29739–29755

    Article  Google Scholar 

  52. Zhu P, Hu Q, Zhang C, Zuo W (2016) Coupled dictionary learning for unsupervised feature selection. In: AAAI conference on artificial intelligence, pp 2422–2428

  53. Zhu P, Zhu W, Hu Q, Zhang C, Zuo W (2017) Subspace clustering guided unsupervised feature selection. Pattern Recogn 66:364–374

    Article  Google Scholar 

  54. Zhu P, Zuo W, Zhang L, Hu Q, Shiu SCK (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446

    Article  MATH  Google Scholar 

  55. Zhu X, Li X, Zhang S, Ju C, Wu X (2017) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Transactions on Neural Networks and Learning Systems 28(6):1263–1275

    Article  MathSciNet  Google Scholar 

  56. Zhu X, Zhu Y, Zhang S, Hu R, He W (2017) Adaptive hypergraph learning for unsupervised feature selection. In: International joint conference on artificial intelligence, pp 3581–3587

Download references

Acknowledgments

This work was supported in part by the National Natural Science Foundation of China under Grant No.61572515 and 61701451, in part by the Fundamental Research Funds for the Central Universities, China University of Geosciences (Wuhan) under Grant No.CUG170654, and in part by China Postdoctoral Science Foundation under Grant No. 2016M593023.

Author information

Authors and Affiliations

  1. School of Statistics, Southwestern University of Finance and Economics, Chengdu, 611130, China

    Deqiong Ding

  2. Associate Research Fellow, Naval University of Engineering; Optical Engineering Postdoctoral Mobile Station, NUDT, Changsha, 410073, China

    Fei Xia

  3. College of Mechanical Engineering, Hunan University of Arts and Science, Changde, 415000, China

    Xiaogao Yang

  4. School of Computer Science, China University of Geosciences, Wuhan, 430074, China

    Chang Tang

Authors
  1. Deqiong Ding
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fei Xia
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiaogao Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Chang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Fei Xia, Xiaogao Yang or Chang Tang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ding, D., Xia, F., Yang, X. et al. Joint dictionary and graph learning for unsupervised feature selection. Appl Intell 50, 1379–1397 (2020). https://doi.org/10.1007/s10489-019-01561-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01561-x

Keywords

Search

Navigation