Skip to main content
SpringerLink
Log in

Robust inner product regularized unsupervised feature selection

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Feature selection aims to select the optimal feature subset which can reduce time complexity, save storage space and improve the performances of various tasks. In this paper, a novel algorithm termed Robust Inner Product Regularized Unsupervised Feature Selection (RIRUFS) is proposed. In RIRUFS algorithm, we firstly combine the self-representation of samples, spectral clustering and feature selection into a unified framework. In this way, RIRUFS can well uncover the underlying multi-subspace structure of the data and iteratively learn the most optimal similarity matrix and clustering labels. Secondly, through introducing the inner product regularization into our objective function, the features selected by our RIRUFS possess the independence and contain low redundancy. Moreover, an effective iterative updating optimization algorithm is developed to solve the RIRUFS model. Extensive experimental results on six datasets show our algorithm is more effective than existing state-of-the-art unsupervised feature selection methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Ahmadian A, Mostafa A (2003) An efficient texture classification algorithm using gabor wavelet. In: Proceedings of the 25th annual international conference of the IEEE engineering in medicine and biology society, 2003, pp 930–933

  2. Boyd S, Vandenberghe L (2004) Convex optimization. Cambridge university press, Los Angeles

    Book  Google Scholar 

  3. Cai D, He X, Hu Y, Han J, Huang T (2007) Learning a spatially smooth subspace for face recognition. In: IEEE Conference on computer vision and pattern recognition, 2007. CVPR’07, pp 1–7

  4. Chen Y, Yi Z (2019) Locality-constrained least squares regression for subspace clustering. Knowl-Based Syst 163:51–56

    Article  Google Scholar 

  5. Dong W, Wu XJ, Kittler J, Yin HF (2019) Sparse subspace clustering via nonconvex approximation. Pattern Anal Appl 22(1):165–176

    Article  MathSciNet  Google Scholar 

  6. Doquire G, Verleysen M (2013) A graph laplacian based approach to semi-supervised feature selection for regression problems. Neurocomputing 121:5–13

    Article  Google Scholar 

  7. Du S, Ma Y, Li S, Ma Y (2017) Robust unsupervised feature selection via matrix factorization. Neurocomputing 241:115–127

    Article  Google Scholar 

  8. Elhamifar E, Vidal R (2013) Sparse subspace clustering: Algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Article  Google Scholar 

  9. Feng Y, Xiao J, Zhuang Y, Liu X (2012) Adaptive unsupervised multi-view feature selection for visual concept recognition. In: Asian conference on computer vision, pp 343–357

    Chapter  Google Scholar 

  10. Freije WA, Castro-Vargas FE, Fang Z, Horvath S, Cloughesy T, Liau L M, Mischel PS, Nelson SF (2004) Gene expression profiling of gliomas strongly predicts survival. Cancer Res 64(18):6503–6510

    Article  Google Scholar 

  11. Gao H, Nie F, Li X, Huang H (2015) Multi-view subspace clustering. In: The IEEE international conference on computer vision (ICCV), pp 4238–4246

  12. Guo J, Quo Y, Kong X, He R (2017) Unsupervised feature selection with ordinal locality. In: 2017 IEEE international conference on multimedia and expo (ICME), pp 1213–1218. IEEE

  13. Han D, Kim J (2015) Unsupervised simultaneous orthogonal basis clustering feature selection. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5016–5023

  14. Han J, Sun Z, Hao H (2015) Selecting feature subset with sparsity and low redundancy for unsupervised learning. Knowl-Based Syst 86:210–223

    Article  Google Scholar 

  15. He X, Cai D, Niyogi P (2005) Laplacian score for feature selection. In: International conference on neural information processing systems, pp 507–514

  16. Hond D, Spacek L (1997) Distinctive descriptions for face processing. In: BMVC, pp 0–4

  17. Hou C, Nie F, Li X, Yi D, Wu Y (2014) Joint embedding learning and sparse regression: A framework for unsupervised feature selection. IEEE Transactions on Cybernetics 44(6):793–804

    Article  Google Scholar 

  18. Ishikawa M, Yoshida K, Yamashita Y, Ota J, Takada S, Kisanuki H, Koinuma K, Choi YL, Kaneda R, Iwao T et al (2005) Experimental trial for diagnosis of pancreatic ductal carcinoma based on gene expression profiles of pancreatic ductal cells. Cancer Sci 96(7):387–393

    Article  Google Scholar 

  19. Jiang H, Robinson DP, Vidal R, You C (2018) A nonconvex formulation for low rank subspace clustering: algorithms and convergence analysis. Computational Optimization and Applications pp 1–24

  20. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556–562

  21. Li Z, Yang Y, Liu J, Zhou X, Lu H, et al (2012) Unsupervised feature selection using nonnegative spectral analysis. In: Association for the advancement of artificial intelligence (AAAI), pp 1026–1032

  22. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  23. Lu C, Feng J, Lin Z, Mei T, Yan S (2019) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Mach Intell 41(2):487–501

    Article  Google Scholar 

  24. Luxburg UV (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  25. Ma B, Xia Y (2017) A tribe competition-based genetic algorithm for feature selection in pattern classification. Appl Soft Comput 58:328–338

    Article  Google Scholar 

  26. Qi M, Wang T, Liu F, Zhang B, Wang J, Yi Y (2018) Unsupervised feature selection by regularized matrix factorization. Neurocomputing 273:593–610

    Article  Google Scholar 

  27. Ranzato M, Szummer M (2008) Semi-supervised learning of compact document representations with deep networks. In: Proceedings of the 25th international conference on Machine learning, pp 792–799. ACM

  28. Samaria FS, Harter AC (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of the 2nd IEEE workshop on applications of computer vision, 1994, pp 138–142

  29. Shang R, Zhang Z, Jiao L, Liu C, Li Y (2016) Self-representation based dual-graph regularized feature selection clustering. Neurocomputing 171:1242–1253

    Article  Google Scholar 

  30. Shang R, Wang W, Stolkin R, Jiao L (2018) Non-negative spectral learning and sparse regression-based dual-graph regularized feature selection. IEEE transactions on cybernetics 48(2):793–806

    Article  Google Scholar 

  31. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22:888–905

    Article  Google Scholar 

  32. Solorio-Fernández S, Carrasco-Ochoa JA, Martínez-Trinidad JF (2019) A review of unsupervised feature selection methods. Artificial Intelligence Review pp 1–42

  33. Spira A, Beane JE, Shah V, Steiling K, Liu G, Schembri F, Gilman S, Dumas YM, Calner P, Sebastiani P et al (2007) Airway epithelial gene expression in the diagnostic evaluation of smokers with suspect lung cancer. Nat Med 13(3):361

    Article  Google Scholar 

  34. Tang C, Liu X, Li M, Wang P, Chen J, Wang L, Li W (2018) Robust unsupervised feature selection via dual self-representation and manifold regularization. Knowl-Based Syst 145:109–120

    Article  Google Scholar 

  35. Wang S, Wang H (2017) Unsupervised feature selection via low-rank approximation and structure learning. Knowl-Based Syst 124:70–79

    Article  Google Scholar 

  36. Wang J, Wu L, Kong J, Li Y, Zhang B (2013) Maximum weight and minimum redundancy: a novel framework for feature subset selection. Pattern Recogn 46(6):1616–1627

    Article  Google Scholar 

  37. Wang S, Tang J, Liu H (2015) Embedded unsupervised feature selection. In: Association for the advancement of artificial intelligence (AAAI), pp 470–476

  38. Yan C, Xie H, Chen J, Zha Z, Hao X, Zhang Y, Dai Q (2018) A fast uyghur text detector for complex background images. IEEE Trans Multimed 20 (12):3389–3398

    Article  Google Scholar 

  39. Yan C, Li L, Zhang C, Liu B, Zhang Y, Dai Q (2019) Cross-modality bridging and knowledge transferring for image understanding. IEEE Transactions on Multimedia

  40. Yang Y, Shen HT, Nie F, Ji R, Zhou X (2011) Nonnegative spectral clustering with discriminative regularization. In: Association for the advancement of artificial intelligence (AAAI), pp 2–4

  41. Zhang Q, Dai J (2018) Cluster structure preserving based on dictionary pair for unsupervised feature selection. In: 2018 International joint conference on neural networks (IJCNN), pp 1–8. IEEE

  42. Zhang H, Zhuang Y, Wu F (2007) Cross-modal correlation learning for clustering on image-audio dataset. In: Proceedings of the 15th ACM international conference on multimedia, pp 273–276

  43. Zhang J, Xie Y, Xia Y, Shen C (2019) Attention residual learning for skin lesion classification. IEEE transactions on medical imaging

  44. Zhao Z, He X, Cai D, Zhang L, Ng W, Zhuang Y (2015) Graph regularized feature selection with data reconstruction. IEEE Trans Knowl Data Eng 28 (3):689–700

    Article  Google Scholar 

  45. Zhou W, Wu C, Yi Y, Luo G (2017) Structure preserving non-negative feature self-representation for unsupervised feature selection. IEEE Access 5:8792–8803

    Article  Google Scholar 

  46. Zhu P, Zuo W, Zhang L, Hu Q, Shiu S C (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446

    Article  Google Scholar 

  47. Zhu P, Zhu W, Hu Q, Zhang C, Zuo W (2017) Subspace clustering guided unsupervised feature selection. Pattern Recogn 66:364–374

    Article  Google Scholar 

  48. Zhu X, Li X, Zhang S, Ju C, Wu X (2017) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Transactions on Neural Networks and Learning Systems 28(6):1263–1275

    Article  MathSciNet  Google Scholar 

Download references

Acknowledgements

We would like to thank Dr. Jianzhong Wang from School of Information Science and Technology, Northeast Normal University for his help with experiment design, technical editing and acquisition of funding during the revision of this paper. This work is supported by the National Natural Science Foundation of China under Grants 11871141 and the Funds for the Central Universities under Grant 2412019FZ049.

Author information

Authors and Affiliations

  1. Key Laboratory for Applied Statistics of MOE, School of Mathematics and Statistics, Northeast Normal University, Changchun, Jilin, 130024, China

    Youcheng Qian & Wei Gao

  2. Applied Mathematics, School of Science, Jilin Institute of Chemical Technology, Jilin, 132022, China

    Youcheng Qian

  3. School of Computer Science and Technology, Dalian University of Technology, Dalian, Liaoning, 116024, China

    Xueyan Yin

Authors
  1. Youcheng Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xueyan Yin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wei Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wei Gao.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Qian, Y., Yin, X. & Gao, W. Robust inner product regularized unsupervised feature selection. Multimed Tools Appl 78, 33593–33615 (2019). https://doi.org/10.1007/s11042-019-08159-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-019-08159-y

Keywords

Search

Navigation