Skip to main content
SpringerLink
Log in

Adaptive graph learning and low-rank constraint for supervised spectral feature selection

  • Multi-Source Data Understanding (MSDU)
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Spectral feature selection (SFS) effectively improves performance of feature selection by introducing a graph matrix to preserve information of data. However, conventional SFS (1) generally preserves either global structure or local structure of data in selected subset, which is not capable of providing comprehensive information for model to output a robust result; (2) constructs graph matrix by original data, which usually lead to a suboptimal graph matrix because of redundant information; (3) conducts feature selection task depending on the fixed graph matrix, which is easily trapped in local optimization. Thus, we have proposed a novel SFS to (1) preserve both local information and global information of original data in feature-selected subset to provide comprehensive information for learning model; (2) integrate graph construction and feature selection to propose a robust spectral feature selection easily obtaining global optimization of feature selection. Besides, for the proposed problem, we further provide a optimization algorithm to effectively tackle the problem with a fast convergence. The extensive experimental results showed that our proposed method outperforms state-of-the-art feature selection methods, in terms of classification performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/.

  2. http://featureselection.asu.edu/datasets.php.

References

  1. Zhu X, Huang Z, Yang Y, Shen HT, Changsheng X, Luo J (2013) Self-taught dimensionality reduction on the high-dimensional small-sized data. Pattern Recognit 46(1):215–229

    MATH  Google Scholar 

  2. Gao L, Song J, Liu X, Shao J, Liu J, Shao J (2017) Learning in high-dimensional multimedia data: the state of the art. Multimed Syst 23(3):303–313

    Google Scholar 

  3. Zhu P, Zuo W, Zhang L, Qinghua H, Shiu SCK (2015) Unsupervised feature selection by regularized self-representation. Pattern Recognit 48(2):438–446

    MATH  Google Scholar 

  4. Nie Feiping, Huang Heng, Cai Xiao, Ding Chris HQ (2010) Efficient and robust feature selection via joint l2,1-norms minimization. In: NIPS, pp 1813–1821

  5. Xiaodong Wang X, Zhang ZZ, Qun W, Zhang J (2016) Unsupervised spectral feature selection with l1-norm graph. Neurocomputing 200:47–54

    Google Scholar 

  6. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: ACM SIGKDD, pp 333–342

  7. Gu Q, Li Z, Han J (2011) Joint feature selection and subspace learning. In: IJCAI, pp 1294–1299

  8. Nie F, Dong X, Tsang WH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. IEEE Trans Image Process 19(7):1921–1932

    MathSciNet  MATH  Google Scholar 

  9. Hinrichs A, Novak E, Ullrich M, Wozniakowski H (2014) The curse of dimensionality for numerical integration of smooth functions ii. J Complex 30(2):117–143

    MathSciNet  MATH  Google Scholar 

  10. Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: ICML, pp 1151–1157

  11. Song J, Gao L, Zou F, Yan Y, Sebe N (2016) Deep and fast: deep learning hashing with semi-supervised graph construction. Image Vis Comput 55:101–108

    Google Scholar 

  12. Zhu X, Li X, Zhang S, Zongben X, Litao Y, Wang C (2017) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044

    Google Scholar 

  13. Yang J, Frangi AF, Yang JY, Zhang D, Jin Z (2005) Kpca plus lda: a complete kernel fisher discriminant framework for feature extraction and recognition. IEEE Trans Pattern Anal Mach Intell 27(2):230

    Google Scholar 

  14. Cai X, Nie F, Huang H (2013) Exact top-k feature selection via l2,0 -norm constraint. In: IJCAI, pp 1240–1246

  15. Zhu X, Zhang S, He W, Hu R, Lei C, Zhu P (2018) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2018.2873378

    Google Scholar 

  16. Zheng W, Zhu X, Wen G, Zhu Y, Yu H, Gan J (2018) Unsupervised feature selection by self-paced learning regularization. Pattern Recognit Lett. https://doi.org/10.1016/j.patrec.2018.06.029

    Google Scholar 

  17. Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: AAAI, pp 1302–1308

  18. Zhu X, Li X, Zhang S, Chunhua J, Xindong W (2017) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans Neural Netw Learn Syst 28(6):1263–1275

    MathSciNet  Google Scholar 

  19. Zhu X, Zhang S, Rongyao H, Zhu Y, Song J (2018) Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans Knowl Data Eng 30(3):517–529

    Google Scholar 

  20. Chang X, Nie F, Yang Y, Huang H (2014) A convex formulation for semi-supervised multi-label feature selection. In: AAAI, pp 1171–1177

  21. Zhu Y, Liang Z, Liu X, Sun K (2017) Self-representation graph feature selection method for classification. Multimed Syst 23(3):1–6

    Google Scholar 

  22. Zhu Y, Zhong Z, Cao W, Cheng D (2016) Graph feature selection for dementia diagnosis. Neurocomputing 195(C):19–22

    Google Scholar 

  23. Zheng W, Zhu X, Zhu Y, Rongyao H, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimed Tools Appl 77(22):29739–29755

    Google Scholar 

  24. Zhu X, Suk H-II, Shen D (2014) Sparse discriminative feature selection for multi-class Alzheimers disease classification. In: MLMI, pp 157–164

  25. Hou C, Nie F, Yi D, Wu Y (2011) Feature selection via joint embedding learning and sparse regression. In: IJCAI, pp 1324–1329

  26. Ding W, Wu X, Zhang S, Zhu X (2013) Feature selection by joint graph sparse coding. In: SIAM, pp 803–811

  27. Lei C, Zhu X (2018) Unsupervised feature selection via local structure learning and sparse learning. Multimed Tools Appl 77(22):29605–29622

    Google Scholar 

  28. Zhu Y, Zhu X, Kim M, Shen D, Wu G (2016) Early diagnosis of Alzheimers disease by joint feature selection and classification on temporally structured support vector machine. In: MICCAI, pp 264–272

  29. Rongyao H, Zhu X, Cheng D, He W, Yan Y, Song J, Zhang S (2017) Graph self-representation method for unsupervised feature selection. Neurocomputing 220:130–137

    Google Scholar 

  30. Liu H, Ma Z, Zhang S, Xindong W (2015) Penalized partial least square discriminant analysis with l1- norm for multi-label data. Pattern Recognit 48(5):1724–1733

    Google Scholar 

  31. Zhu X, Suk H-I, Wang L, Lee S-W, Shen D (2017) A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Medical Image Anal 38:205–214

    Google Scholar 

  32. Zhu X, Zhang L, Huang Z (2014) A sparse embedding and least variance encoding approach to hashing. IEEE Trans Image Process 23(9):3737–3750

    MathSciNet  MATH  Google Scholar 

  33. Zhu X, Suk H-II, Huang H, Shen D (2017) Low-rank graph-regularized structured sparse regression for identifying genetic biomarkers. IEEE Trans Big Data. https://doi.org/10.1109/TBDATA.2017.2735991

    Google Scholar 

  34. Cai X, Ding C, Nie F, Huang H (2013) On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: ACM SIGKDD, pp 1124–1132

  35. Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2018) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng PP(99):1–1

    Google Scholar 

  36. Zhang S, Li X, Zong M, Zhu X, Cheng D (2017) Learning k for knn classification. Acm Trans Intell Syst Technol 8(3):43

    Google Scholar 

  37. Zhang S, Li X, Zong M, Zhu X, Wang R (2018) Efficient knn classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst 29(5):1774–1785

    MathSciNet  Google Scholar 

  38. Daubechies I, DeVore RA, Fornasier M, Gunturk CS (2008) Iteratively re-weighted least squares minimization: proof of faster than linear rate for sparse recovery. In: CISS, pp 26–29

  39. Boyd S, Vandenberghe L, Faybusovich L (2006) Convex optimization. IEEE Trans Autom Control 51(11):1859–1859

    Google Scholar 

  40. Qian M, Zhai C (2013) Robust unsupervised feature selection. In: IJCAI, pp 1621–1627

Download references

Acknowledgements

This work was supported by the program of Research and development of intelligent logistics management system based on Beidou multi-functional information acquisition and monitoring terminal (Grant No: 2016AB04097).

Author information

Authors and Affiliations

  1. College of Continuing Education, Guangxi Teachers Education University, Nanning, 530001, Guangxi, People’s Republic of China

    Zhi Zhong

Authors
  1. Zhi Zhong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhi Zhong.

Ethics declarations

Conflict of interest

We wish to draw the attention of the Editor to the following facts which may be considered as potential conflicts of interest and to significant financial contributions to this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhong, Z. Adaptive graph learning and low-rank constraint for supervised spectral feature selection. Neural Comput & Applic 32, 6503–6512 (2020). https://doi.org/10.1007/s00521-018-04006-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-018-04006-7

Keywords

Search

Navigation