Skip to main content
SpringerLink
Log in

Unsupervised feature selection via local structure learning and sparse learning

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Feature self-representation has become the backbone of unsupervised feature selection, since it is almost insensitive to noise data. However, feature selection methods based on feature self-representation have the following drawbacks: 1) The self-representation coefficient matrix is fixed and can not be fine-tuned according to the structure of data. 2) they do not consider the manifold structure of data, thus unable to further increase the performance of feature selection. To solve the above problems, this paper proposes an unsupervised feature selection algorithm that combines feature self-representation and manifold learning. Specifically, we first utilize feature self-representation to construct the model. After that, the self-representation coefficient matrix is dynamically adjusted to the optimal state based on the similarity matrix. Then, we use low-rank representation to explore the global manifold structure of the data. Finally, we combine sparse learning with feature selection. The experimental results on twelve datasets show that the proposed method outperforms all the competing methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. 1 http://archive.ics.uci.edu/ml/.

  2. 2 http://featureselection.asu.edu/datasets.php.

References

  1. Boyd S, Vandenberghe L (2013) Convex optimization

  2. Cai D, He X, Han J (2007) Spectral regression: a unified approach for sparse subspace learning. In: ICDM, pp 73–82

  3. Cai D, Zhang C, He X (2010) Unsupervised feature selection for multi-cluster data. In: KDD, pp 333–342

  4. Cai X, Nie F, Huang H (2013) Exact top-k feature selection via l 2,0 -norm constraint. In: IJCAI, pp 1240–1246

  5. Chang X, Nie F, Yi Y, Huang H (2014) A convex formulation for semi-supervised multi-label feature selection. In: AAAI, pp 1171–1177

  6. Daubechies I, Devore R, Fornasier M, SiNan Gntk C (2008) Iteratively reweighted least squares minimization for sparse recovery. Commun Pure Appl Math 63(1):1–38

    Article  MathSciNet  MATH  Google Scholar 

  7. Fan Z, Yong X, Zhang D (2011) Local linear discriminant analysis framework using sample neighbors. IEEE Trans Neural Netw 22(7):1119–1132

    Article  Google Scholar 

  8. Gao S, Tsang I W, Chia L-T (2013) Sparse representation with kernels. IEEE Trans Image Process 22(2):423–434

    Article  MathSciNet  MATH  Google Scholar 

  9. Gao L, Song J, Liu X, Shao J, Liu J, Shao J (2017) Learning in high-dimensional multimedia data: the state of the art. Multimed Syst 23(3):303–313

    Article  Google Scholar 

  10. Gao L, Wang Y, Li D, Shao J, Song J (2017) Real-time social media retrieval with spatial, temporal and social constraints. Neurocomputing 253:77–88

    Article  Google Scholar 

  11. Hu R, Zhu X, Cheng D, He W, Yan Y, Song J, Zhang S (2017) Graph self-representation method for unsupervised feature selection. Neurocomputing 220:130–137

    Article  Google Scholar 

  12. Jayasena K P N, Li L, Xie Q (2017) Multi-modal multimedia big data analyzing architecture and resource allocation on cloud platform. Neurocomputing

  13. Ling C X, Yang Q, Wang J, Zhang S (2004) Decision trees with minimal costs. In: ICML, pp 69

  14. Nie F, Zhu W, Li X (2016) Unsupervised feature selection with structured graph optimization. In: AAAI, pp 1302–1308

  15. Qian B, Wang X, Cao N, Gang Jiang Y, Davidson I (2014) Learning multiple relative attributes with humans in the loop. IEEE Trans Image Process 23 (12):5573–5585

    Article  MathSciNet  MATH  Google Scholar 

  16. Qian B, Wang X, Cao N, Li H, Gang Jiang Y (2015) A relative similarity based method for interactive patient risk prediction. Data Mining Knowl Discov 29 (4):1070–1093

    Article  MathSciNet  Google Scholar 

  17. Qin Y, Zhang S, Zhu X, Zhang J, Zhang C (2007) Semi-parametric optimization for missing data imputation. Appl Intell 27(1):79–88

    Article  MATH  Google Scholar 

  18. Song J, Yi Y, Zi H, Shen H T, Luo J (2013) Effective multiple feature hashing for large-scale near-duplicate video retrieval. IEEE Trans Multimed 15 (8):1997–2008

    Article  Google Scholar 

  19. Song J, Gao L, Nie F, Shen H T, Yan Y, Sebe N (2016) Optimized graph learning using partial tags and multiple features for image and video annotation. IEEE Trans Image Process 25(11):4999–5011

    Article  MathSciNet  Google Scholar 

  20. Song J, Gao L, Zou F, Yan Y, Sebe N (2016) Deep and fast: deep learning hashing with semi-supervised graph construction. Image Vis Comput 55:101–108

    Article  Google Scholar 

  21. Song J, Shen H T, Wang J, Zi H, Sebe N, Wang J (2016) A distance-computation-free search scheme for binary code databases. IEEE Trans Multimed 18(3):484–495

    Article  Google Scholar 

  22. Sun J, Zhou A (2014) Unsupervised robust bayesian feature selection, pp 558–564

  23. Wang T, Qin Z, Zhang S, Zhang C (2012) Cost-sensitive classification with inadequate labeled data. Inf Syst 37(5):508–516

    Article  Google Scholar 

  24. Wang X, Qian B, Davidson I (2012) On constrained spectral clustering and its applications. Data Mining Knowl Discov 28(1):1–30

    Article  MathSciNet  MATH  Google Scholar 

  25. Wang D, Nie F, Huang H (2014) Unsupervised feature selection via unified trace ratio formulation and k-means clustering (track). In: Ecml/pkdd, pp 306–321

    Google Scholar 

  26. Wen Z, Yin W (2013) A feasible method for optimization with orthogonality constraints. Math Program 142(1):397–434

    Article  MathSciNet  MATH  Google Scholar 

  27. Xia Y, He K, Kohli P, Sun J (2015) Sparse projections for high-dimensional binary codes. In: Computer vision and pattern recognition, pp 3332–3339

  28. Xie Q, Pang C, Zhou X, Zhang X, Ke D (2014) Maximum error-bounded piecewise linear representation for online stream approximation. Vldb J 23(6):915–937

    Article  Google Scholar 

  29. Xie QS, Wang JZ, Zhang X (2016) Modeling and predicting ad progression by regression analysis of sequential clinical data. Neurocomputing 195(C):50–55

    Article  Google Scholar 

  30. Xie Q, Zhang X, Li Z, Zhou X (2016) Optimizing cost of continuous overlapping queries over data streams by filter adaption. IEEE Trans Knowl Data Eng 28(5):1258–1271

    Article  Google Scholar 

  31. Xindong W, Zhang S (2003) Synthesizing high-frequency rules from different data sources. IEEE Trans Knowl Data Eng 15(2):353–367

    Article  Google Scholar 

  32. Xindong W, Zhang C, Zhang S (2004) Efficient mining of both positive and negative association rules. Acm Trans Inf Syst 22(3):381–405

    Article  Google Scholar 

  33. Xindong W, Zhang C, Zhang S (2005) Database classification for multi-database mining. Inf Syst 30(1):71–88

    Article  MATH  Google Scholar 

  34. Yan X, Zhang C, Zhang S (2009) Genetic algorithm-based strategy for identifying association rules without specifying actual minimum support. Expert Syst Appl 36(2):3066–3076

    Article  Google Scholar 

  35. Zhang S (2011) Shell-neighbor method and its application in missing data imputation. Appl Intell 35(1):123–133

    Article  Google Scholar 

  36. Zhang S (2012) Nearest neighbor selection for iteratively knn imputation. J Syst Softw 85(11):2541–2552

    Article  Google Scholar 

  37. Zhang C, Zhang S (2002) Association rule mining: models and algorithms 2307

  38. Zhang S, Zhang C (2002) Anytime mining for multiuser applications. IEEE Trans Syst Man Cybern-Part Syst Humans 32(4):515–521

    Article  Google Scholar 

  39. Zhang S, Zhang C, Yang Q (1999) Data preparation for data mining. Academic Press

  40. Zhang S, Zhang C, Yan X (2003) Post-mining: maintenance of association rules by weighting. Inf Syst 28(7):691–707

    Article  Google Scholar 

  41. Zhang S, Wu X, Zhang C (2003) Multi-database mining 2:5–13

    Google Scholar 

  42. Zhang S, Qin Z, Ling C X, Sheng S (2005) Missing is useful?: missing values in cost-sensitive decision trees. IEEE Trans Knowl Data Eng 17(12):1689–1693

    Article  Google Scholar 

  43. Zhang S, Jin Z, Zhu X (2011) Missing data imputation by utilizing information within incomplete instances. J Syst Softw 84(3):452–459

    Article  Google Scholar 

  44. Zhang S, Li X, Zong M, Zhu X, Cheng D (2017) Learning k for knn classification. ACM Trans Intell Syst Technol 8(3):43

    Google Scholar 

  45. Zhang S, Li X, Zong M, Zhu X, Wang R (2017) Efficient knn classification with different numbers of nearest neighbors. IEEE Trans Neural Netw Learn Syst. https://doi.org/10.1109/TNNLS.2017.2673241

    Article  MathSciNet  Google Scholar 

  46. Zhao Y, Zhang S (2005) Generalized dimension-reduction framework for recent-biased time series analysis. IEEE Trans Knowl& Data Eng 18(2):231–244

    Article  Google Scholar 

  47. Zhong F, Zhang J (2013) Linear discriminant analysis based on l1-norm maximization. IEEE Trans Image Process 22(8):3018–3027

    Article  MathSciNet  MATH  Google Scholar 

  48. Zhu Y, Lucey S (2015) Convolutional sparse coding for trajectory reconstruction. IEEE Trans Pattern Anal Mach Intell 37(3):529–540

    Article  Google Scholar 

  49. Zhu X, Zhang S, Jin Z, Zhang Z (2011) Missing value estimation for mixed-attribute data sets. IEEE Trans Knowl Data Eng 23(1):110–121

    Article  Google Scholar 

  50. Zhu X, Zi H, Shen H T, Cheng J, Changsheng X (2012) Dimensionality reduction by mixed kernel canonical correlation analysis. Pattern Recogn 45(8):3003–3016

    Article  MATH  Google Scholar 

  51. Zhu X, Zi H, Shen H T, Zhao X (2013) Linear cross-modal hashing for efficient multimedia search. In: ACM International conference on multimedia, pp 143–152

  52. Zhu X, Zi H, Cheng H, Cui J, Shen H T (2013) Sparse hashing for fast multimedia search. ACM Trans Inf Syst 31(2):9

    Article  Google Scholar 

  53. Zhu X, Zi H, Yang Y, Shen H T, Changsheng X, Luo J (2013) Self-taught dimensionality reduction on the high-dimensional small-sized data. Pattern Recogn 46(1):215–229

    Article  MATH  Google Scholar 

  54. Zhu X, Zhang L, Huang Z (2014) A sparse embedding and least variance encoding approach to hashing. IEEE Trans Image Process 23(9):3737

    Article  MathSciNet  MATH  Google Scholar 

  55. Zhu X, Suk H I, Shen D (2014) A novel matrix-similarity based loss function for joint regression and classification in ad diagnosis. Neuroimage 100:91–105

    Article  Google Scholar 

  56. Zhu P, Zuo W, Zhang L, Qinghua H, Shiu S C (2015) Unsupervised feature selection by regularized self-representation. Pattern Recogn 48(2):438–446

    Article  MATH  Google Scholar 

  57. Zhu X, Xie Q, Zhu Y, Liu X, Zhang S (2015) Multi-view multi-sparsity kernel reconstruction for multi-class image classification. Neurocomputing 169:43–49

    Article  Google Scholar 

  58. Zhu X, Suk H I, Lee S W, Shen D (2015) Canonical feature selection for joint regression and multi-class identification in alzheimer’s disease diagnosis. Brain Imag Behav 10(3):1–11

    Google Scholar 

  59. Zhu X, Li X, Zhang S (2016) Block-row sparse multiview multilabel learning for image classification. IEEE Trans Cybern 46(2):450

    Article  Google Scholar 

  60. Zhu X, Suk H-I, Lee S-W, Shen D (2016) Subspace regularized sparse multitask learning for multiclass neurodegenerative disease identification. IEEE Trans Biomed En. 63(3):607–618

    Article  Google Scholar 

  61. Zhu Y, Zhu X, Kim M, Shen D, Guorong W (2016) Early diagnosis of alzheimers disease by joint feature selection and classification on temporally structured support vector machine. In: MICCAI, pp 264–272

  62. Zhu X, He W, Li Y, Yang Y, Zhang S, Rongyao H, Zhu Y (2017) One-step spectral clustering via dynamically learning affinity matrix and subspace. In: AAAI, pp 2963–2969

  63. Zhu X, Li X, Zhang S, Chunhua J, Xindong W (2017) Robust joint graph sparse coding for unsupervised spectral feature selection. IEEE Trans Neural Netw Learn Syst 28(6):1263–1275

    Article  MathSciNet  Google Scholar 

  64. Zhu X, Li X, Zhang S, Xu Z, Yu L, Wang C (2017) Graph PCA hashing for similarity search. IEEE Multimed Multimed 19(9):2033–2044

    Article  Google Scholar 

  65. Zhu X, Suk HII, Huang H, Shen D (2017) Low-rank graph-regularized structured sparse regression for identifying genetic biomarkers. IEEE Transact Big Data. https://doi.org/10.1109/TBDATA.2017.2735991

    Article  Google Scholar 

  66. Zhu X, Suk H-I, Wang L, Lee S-W, Shen D (2017) A novel relational regularization feature selection method for joint regression and classification in AD diagnosis. Med Image Anal 38:205–214

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported in part by the China Key Research Program (Grant No: 2016YFB1000905), the China 1000-Plan National Distinguished Professorship, the Nation Natural Science Foundation of China (Grants No: 61573270, 61672177 and 61363009), the Guangxi Natural Science Foundation (Grant No: 2015GXNSFCB139-011), the Guangxi High Institutions Program of Introducing 100 High-Level Overseas Talents, the Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing, the Guangxi Bagui Teams for Innovation and Research, the Research Fund of Guangxi Key Lab of MIMS (16-A-01-01 and 16-A-01-02), the Guangxi Bagui Teams for Innovation and Research, and Innovation Project of Guangxi Graduate Education under grant XYCSZ2017064, XYCSZ2017067 and YCSW2017065.

Author information

Authors and Affiliations

  1. Guangxi Key Lab of Multi-source Information Mining and Security, Guangxi Normal University, Guilin, 541004, Guangxi, People’s Republic of China

    Cong Lei & Xiaofeng Zhu

Authors
  1. Cong Lei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaofeng Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaofeng Zhu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lei, C., Zhu, X. Unsupervised feature selection via local structure learning and sparse learning. Multimed Tools Appl 77, 29605–29622 (2018). https://doi.org/10.1007/s11042-017-5381-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-017-5381-7

Keywords

Search

Navigation