Abstract
Unsupervised feature selection is a powerful tool to process high-dimensional data, in which a subset of features is selected out for effective data representation. In this paper, we proposes a novel robust unsupervised features selection method based on graph-preserving feature selection embedding LLE. Specifically, we integrate the graph matrix learning and the low-dimensional space learning together to identify the correlation among both features and samples from the intrinsic low-dimensional space of original data. Also, the global and local correlation of features have been taken into consideration through the low-rank constraint and the feature-level representation property to find lower-dimensional representation which preserves not only the global and local correlation of features but also the global and local structure of training samples. Furthermore, we propose a new optimization algorithm to the resulting objective function, which iteratively updates the graph matrix and the intrinsic space in order to collaboratively improve each of them. Experimental analysis on 18 benchmark datasets verified that our proposed method outperformed the state-of-the-art feature selection methods in terms of classification and clustering performance.
Similar content being viewed by others
References
Benabdeslem K, Hindawi M (2014) Efficient semi-supervised feature selection: constraint, relevance, and redundancy. IEEE Trans Knowl Data Eng 26(5):1131–1143
Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, pp 333–342
Cai X, Ding C, Nie F et al (2013) On the equivalent of low-rank linear regressions and linear discriminant analysis based regressions. In: Proceedings of the 19th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 1124–1132
Chen L, Huang JZ (2012) Sparse reduced-rank regression for simultaneous dimension reduction and variable selection. J Am Stat Assoc 107(500):1533–1545
Cheng D, Zhang S, Liu X et al (2017) Feature selection by combining subspace learning with sparse representation. Multimed Syst 23(3):285–291
Du L, Shen YD (2015) Unsupervised feature selection with adaptive structure learning. In: Proceedings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining. ACM, pp 209–218
Du L, Shen Z, Li X et al (2013) Local and global discriminative learning for unsupervised feature selection. In: 2013 IEEE 13th international conference on data mining (ICDM). IEEE, pp 131–140
Du S, Wang W, Ma Y (2016) Low rank sparse preserve projection for face recognition. In: Control and decision conference (CCDC), 2016 Chinese. IEEE, pp 3822–3826
Gao S, Ver Steeg G, Galstyan A (2016) Variational information maximization for feature selection. In: Advances in neural information processing systems, pp 487–495
García-Torres M, Gómez-Vela F, Melián-Batista B et al (2016) High-dimensional feature selection via feature grouping: a variable neighborhood search approach. Inf Sci 326:102–118
Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3(3):1157–1182
Han Y, Xu Z, Ma Z et al (2013) Image classification with manifold learning for out-of-sample data. Signal Process 93(8):2169–2177
He X, Cai D, Niyogi P (2006) Laplacian score for feature selection. In: Weiss Y, Schölkopf B, Platt JC (eds) Advances in neural information processing systems. Neural information processing systems foundation, British Columbia, pp 507–514
Jian L, Li J, Shu K et al (2016) Multi-label informed feature selection. In: IJCAI, pp 1627–1633
Jiang Y, Ren J (2011) Eigenvalue sensitive feature selection. In: Proceedings of the 28th international conference on machine learning (ICML-11), pp 89–96
Lan X, Ma AJ, Yuen PC et al (2015) Joint sparse representation and robust feature-level fusion for multi-cue visual tracking. IEEE Trans Image Process 24(12):5826–5841
Lan X, Zhang S, Yuen PC (2016) Robust joint discriminative feature learning for visual tracking. In: IJCAI, pp 3403–3410
Li Z, Yang Y, Liu J et al (2012) Unsupervised feature selection using nonnegative spectral analysis. In: AAAI, vol 2, pp 1026–1032
Li J, Tang J, Liu H (2017a) Reconstruction-based unsupervised feature selection: an embedded approach. In: Proceedings of the 26th international joint conference on artificial intelligence. IJCAI/AAAI
Li J, Wu L, Zaïane OR et al (2017b) Toward personalized relational learning. In: Proceedings of the 2017 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, pp 444–452
Liu M, Zhang D (2014) Sparsity score: a novel graph-preserving feature selection method. Int J Pattern Recognit Artif Intell 28(04):1450009
Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 663–670
Luo M, Nie F, Chang X et al (2018a) Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst 29(4):944–956
Luo M, Chang X, Nie L et al (2018b) An adaptive semisupervised feature analysis for video semantic recognition. IEEE Trans Cybern 48(2):648–660
Ma J, Zhou H, Zhao J et al (2015) Robust feature matching for remote sensing image registration via locally linear transforming. IEEE Trans Geosci Remote Sens 53(12):6469–6481
Mitra S, Kundu PP, Pedrycz W (2012) Feature selection using structural similarity. Inf Sci 198:48–61
Nie F, Huang H, Cai X et al (2010) Efficient and robust feature selection via joint ℓ2, 1-norms minimization. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems. DBLP, British Columbia, pp 1813–1821
Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM. pp 977–986
Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: AAAI, pp 1302–1308
Peng Y, Long X, Lu BL (2015) Graph based semi-supervised learning via structure preserving low-rank representation. Neural Process Lett 41(3):389–406
Qiao L, Chen S, Tan X (2010) Sparsity preserving projections with applications to face recognition. Pattern Recogn 43(1):331–341
Sheikhpour R, Sarram MA, Gharaghani S et al (2017) A survey on semi-supervised feature selection methods. Pattern Recogn 64:141–158
Shi L, Du L, Shen YD (2014) Robust spectral learning for unsupervised feature selection. In: 2014 IEEE international conference on data mining (ICDM). IEEE, pp 977–982
Shi X, Guo Z, Lai Z et al (2015) A framework of joint graph embedding and sparse regression for dimensionality reduction. IEEE Trans Image Process 24(4):1341–1355
Tang J, Hu X, Gao H et al (2014) Discriminant analysis for unsupervised feature selection. In: Proceedings of the 2014 SIAM international conference on data mining. Society for Industrial and Applied Mathematics, pp 938–946
Tenenbaum JB, De Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323
Wang D, Nie F, Huang H (2014) Unsupervised feature selection via unified trace ratio formulation and k-means clustering (track). In: Joint European conference on machine learning and knowledge discovery in databases. Springer, Berlin, pp 306–321
Wang S, Tang J, Liu H (2015) Embedded unsupervised feature selection. In: AAAI, Citeseer, pp 470–476
Wei X, Philip SY (2016) Unsupervised feature selection by preserving stochastic neighbors. In: Gretton A, Robert CC (eds) Artificial intelligence and statistics. PMLR, Cadiz, pp 995–1003
Yang Y, Zhuang YT, Wu F et al (2008) Harmonizing hierarchical manifolds for multimedia document semantics understanding and cross-media retrieval. IEEE Trans Multimed 10(3):437–446
Yao C, Liu YF, Jiang B et al (2017) LLE score: a new filter-based unsupervised feature selection method based on nonlinear manifold embedding and its application to image recognition. IEEE Trans Image Process 26(11):5257–5269
Zhang L, Song M, Yang Y et al (2014) Weakly supervised photo cropping. IEEE Trans Multimed 16(1):94–107
Zhang L, Gao Y, Xia Y et al (2015) A fine-grained image categorization system by cellet-encoded spatial pyramid modeling. IEEE Trans Industr Electron 62(1):564–571
Zhang D, Han J, Jiang L et al (2017a) Revealing event saliency in unconstrained video collection. IEEE Trans Image Process 26(4):1746–1758
Zhang Y, Wang Y, Jin J et al (2017b) Sparse Bayesian learning for obtaining sparsity of EEG frequency bands based feature vectors in motor imagery classification. Int J Neural Syst 27(02):1650032
Zhang S, Li X, Zong M et al (2017c) Learning k for knn classification. ACM Trans Intell Syst Technol 8(3):43
Zhao Z, Wang L, Liu H (2010) Efficient spectral feature selection with minimum redundancy. In: AAAI, pp 673–678
Zhao Z, Wang L, Liu H et al (2013) On similarity preserving feature selection. IEEE Trans Knowl Data Eng 25(3):619–632
Zhu X, Zhang L, Huang Z (2014) A sparse embedding and least variance encoding approach to hashing. IEEE Trans Image Process 23(9):3737–3750
Zhu P, Zuo W, Zhang L et al (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446
Zhu X, Li X, Zhang S (2016) Block-row sparse multiview multilabel learning for image classification. IEEE Trans Cybern 46(2):450–461
Zhu X, Li X, Zhang S et al (2017a) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044
Zhu X, Li X, Zhang S et al (2017b) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans Neural Netw Learn Syst 28(6):1263–1275
Zhu Y, Zhu X, Kim M et al (2017c) A novel dynamic hyper-graph inference framework for computer assisted diagnosis of neuro-diseases. In: International conference on information processing in medical imaging. Springer, Cham, pp 158–169
Zhu X, Suk HI, Wang L et al (2017d) A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med Image Anal 38:205–214
Zhu X, Zhang S, Hu R et al (2018) Local and global structure preservation for robust unsupervised spectral feature selection. IEEE Trans Knowl Data Eng 30(3):517–529
Acknowledgements
This work was supported by the Natural Science Foundation of Shanxi Province, China (Grant No. 201801D121136) and the Nation Natural Science Foundation of China (Grants Nos. 61872260 and 61772358).
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Han, X., Chai, H., Liu, P. et al. A new graph-preserving unsupervised feature selection embedding LLE with low-rank constraint and feature-level representation. Artif Intell Rev 53, 2875–2903 (2020). https://doi.org/10.1007/s10462-019-09749-w
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10462-019-09749-w