Skip to main content
SpringerLink
Log in

Fast unsupervised feature selection with anchor graph and ℓ2,1-norm regularization

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Graph-based unsupervised feature selection has been proven to be effective in dealing with unlabeled and high-dimensional data. However, most existing methods face a number of challenges primarily due to their high computational complexity. In light of the ever-increasing size of data, these approaches tend to be inefficient in dealing with large-scale data sets. We propose a novel approach, called Fast Unsupervised Feature Selection (FUFS), to efficiently tackle this problem. Firstly, an anchor graph is constructed by means of a parameter-free adaptive neighbor assignment strategy. Meanwhile, an approximate nearest neighbor search technique is introduced to speed up the anchor graph construction. The ℓ2,1-norm regularization is then performed to select more valuable features. Experiments on several large-scale data sets demonstrate the effectiveness and efficiency of the proposed method.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: Proceedings of the conference on knowledge discovery and data mining, pp 333–342

  2. Cheng Q, Zhou H, Cheng J (2011) The fisher-markov selector: fast selecting maximally separable feature subset for multiclass classification with applications to high-dimensional data. IEEE Trans Pattern Anal Mach Intell 33(6):1217–33

    Article  Google Scholar 

  3. Deng C, Ji R, Liu W, Tao D, Gao X (2013) Visual reranking through weakly supervised multi-graph learning. In: Proceedings of the 2013 IEEE international conference on computer vision, pp 2600–2607

  4. Deng C, Ji R, Tao D, Gao X, Li X (2014) Weakly supervised multi-graph learning for robust image reranking. IEEE Trans Multimed 16(3):785–795

    Article  Google Scholar 

  5. Dy JG, Brodley CE (2004) Feature selection for unsupervised learning. J Mach Learn Res 5(4):845–889

    MathSciNet  MATH  Google Scholar 

  6. Freeman C, Kulic D, Basir O (2013) Feature-selected tree-based classification. IEEE Trans Cybern 43(6):1990–2004

    Article  Google Scholar 

  7. Guyon I, Elisseeff A (2003) An introduction to variable and feature selection. J Mach Learn Res 3:1157–1182

    MATH  Google Scholar 

  8. Hancock T, Mamitsuka H (2012) Boosted network classifiers for local feature selection. IEEE Trans Neural Netw Learn Syst 23(11):1767–1778

    Article  Google Scholar 

  9. He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: Proceedings of the conference on neural information processing systems, pp 507–514

  10. Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: a framework for unsupervised feature selection. IEEE Trans Cybern 44(6):793

    Article  Google Scholar 

  11. Hou C, Nie F, Tao H, Yi D (2017) Multi-view unsupervised feature selection with adaptive similarity and view weight. IEEE Trans Knowl Data Eng 29(9):1998–2011

    Article  Google Scholar 

  12. Kokiopoulou E, Saad Y (2007) Orthogonal neighborhood preserving projections: a projection-based dimensionality reduction technique. IEEE Trans Pattern Anal Mach Intell 29(12):2143–2156

    Article  Google Scholar 

  13. Lai HJ, Pan Y, Tang Y, Yu R (2013) Fsmrank: feature selection algorithm for learning to rank. IEEE Trans Neural Netw Learn Syst 24(6):940–952

    Article  Google Scholar 

  14. Laporte L, Flamary R, Canu S, Djean S, Mothe J (2014) Nonconvex regularizations for feature selection in ranking with sparse svm. IEEE Trans Neural Netw Learn Syst 25(6):1118–1130

    Article  Google Scholar 

  15. Li Z, Yang Y, Liu J, Zhou X, Lu H (2012) Unsupervised feature selection using nonnegative spectral analysis. In: Proceedings of the AAAI conference on artificial intelligence, pp 1026–1032

  16. Li Y, Si J, Zhou G, Huang S, Chen S (2015) Frel: a stable feature selection algorithm. IEEE Trans Neural Netw Learn Syst 26(7):1388

    Article  MathSciNet  Google Scholar 

  17. Ling X, Qiang MA, Min Z (2013) Tensor semantic model for an audio classification system. Sci Chin 56(6):1–9

    MathSciNet  Google Scholar 

  18. Liu W, He J, Chang SF (2010) Large graph construction for scalable semi-supervised learning. In: Proceedings of the international conference on machine learning, pp 679–686

  19. Liu W, Wang J, Chang SF (2012) Robust and scalable graph-based semisupervised learning. Proc IEEE 100(9):2624–2638

    Article  Google Scholar 

  20. Luo M, Nie F, Chang X, Yi Y, Hauptmann AG, Zheng Q (2017) Adaptive unsupervised feature selection with structure regularization. IEEE Trans Neural Netw Learn Syst PP(99):1–13

    Google Scholar 

  21. Muja M, Lowe DG (2014) Scalable nearest neighbor algorithms for high dimensional data. IEEE Trans Pattern Anal Mach Intell 36(11):2227–2240

    Article  Google Scholar 

  22. Nie F, Huang H, Cai X, Ding CH (2010) Efficient and robust feature selection via joint ℓ 2,1-norms minimization. In: Proceedings of the conference on advances in neural information processing systems, pp 1813–1821

  23. Nie F, Xiang S, Jia Y, Zhang C, Yan S (2008) Trace ratio criterion for feature selection. In: Proceedings of the AAAI conference on artificial intelligence, pp 671–676

  24. Nie F, Wang X, Huang H (2014) Clustering and projected clustering with adaptive neighbors. In: Proceedings of the conference on knowledge discovery and data mining, pp 977–986

  25. Nie F, Wang X, Jordan M, Huang H (2016) The constrained laplacian rank algorithm for graph-based clustering. In: Proceedings of the AAAI conference on artificial intelligence, pp 1969–1976

  26. Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: Proceedings of the AAAI conference on artificial intelligence, pp 1302–1308

  27. Peng Y, Lu BL (2017) Discriminative extreme learning machine with supervised sparsity preserving for image classification. Neurocomputing

  28. Qian M, Zhai C (2013) Robust unsupervised feature selection. In: Proceedings of the international joint conference on artificial intelligence, pp 1621–1627

  29. Romero E, Sopena JM (2008) Performing feature selection with multilayer perceptrons. IEEE Trans Neural Netw 19(3):431–41

    Article  Google Scholar 

  30. Strehl A, Ghosh J (2002) Cluster ensembles: a knowledge reuse framework for combining partitionings. In: Eighteenth national conference on artificial intelligence, pp 93–98

  31. Wang R, Nie F, Yang X, Gao F, Yao M (2015) Robust 2DPCA with non-greedy ℓ 1-norm maximization for image analysis. IEEE Trans Cybern 45(5):1108–1112

    Article  Google Scholar 

  32. Wang R, Nie F, Hong R, Chang X, Yang X, Yu W (2017) Fast and orthogonal locality preserving projections for dimensionality reduction. IEEE Trans Image Process 26(10):5019–5030

    Article  MathSciNet  Google Scholar 

  33. Xiang S, Nie F, Zhang C, Zhang C (2009) Nonlinear dimensionality reduction with local spline embedding. IEEE Trans Knowl Data Eng 21(9):1285–1298

    Article  Google Scholar 

  34. Xing L, Dong H, Jiang W, Tang K (2017) Nonnegative matrix factorization by joint locality-constrained and ℓ 2,1-norm regularization. Multimed Tools Appl. https://doi.org/10.1007/s11042-017-4970-9

  35. Yang Y, Shen HT, Ma Z, Huang Z, Zhou X (2011) ℓ 2,1-norm regularized discriminative feature selection for unsupervised learning., In: Proceedings of the international joint conference on artificial intelligence, pp 1589–1594

  36. Yu Q, Wang R, Yang X, Li BN, Yao M (2016) Diagonal principal component analysis with non-greedy ℓ 1-norm maximization for face recognition. Neurocomputing 171:57–62

    Article  Google Scholar 

  37. Zhao Z, Liu H (2007) Spectral feature selection for supervised and unsupervised learning. In: Proceedings of the international conference on machine learning, pp 1151–1157

Download references

Acknowledgements

This paper is supported in part by the National Natural Science Foundation of China under Grant 61401471, Grant 61772427 and Grant 61751202 and in part by the China Postdoctoral Science Foundation under Grant 2014M562636.

Author information

Authors and Affiliations

  1. The Xi’an Research Institute of Hi-Tech, Xi’an, 710025, China

    Haojie Hu & Rong Wang

  2. The Center for OPTical IMagery Analysis and Learning (OPTIMAL), Northwestern Polytechnical University, Xi’an, 710072, China

    Rong Wang & Feiping Nie

  3. The School of Information Engineering, Guangdong University of Technology, Guangzhou, 510006, China

    Xiaojun Yang

  4. The School of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an, 710049, China

    Weizhong Yu

Authors
  1. Haojie Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rong Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Feiping Nie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaojun Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Weizhong Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rong Wang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hu, H., Wang, R., Nie, F. et al. Fast unsupervised feature selection with anchor graph and ℓ2,1-norm regularization. Multimed Tools Appl 77, 22099–22113 (2018). https://doi.org/10.1007/s11042-017-5582-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5582-0

Keywords

Search

Navigation