Skip to main content
SpringerLink
Log in

Using Locality Preserving Projections to Improve the Performance of Kernel Clustering

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Many clustering methods may have poor performance when the data structure is complex (i.e., the data has an aspheric shape or non-linear relationship). Inspired by this view, we proposed a clustering model which combines kernel function and Locality Preserving Projections (LPP) together. Specifically, we map original data into the high-dimensional feature space according to the idea of kernel function. Secondly, it is feasible to explore the local structure of data in clustering tasks. LPP is used to preserve the original local structure information of data to improve the validity of the clustering model. Finally, some outliers are often included in real data, so we embedded sparse regularization items in the model to adjust feature weights and remove outliers. In addition, we design a simple iterative optimization method to solve the final objective function and show the convergence of the optimization method in the experimental part. The experimental analysis of ten public data sets showed that our proposed method has better efficiency and performance in clustering tasks than existing clustering methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. http://archive.ics.uci.edu/ml/.

References

  1. Aït-Sahalia Y, Xiu D (2019) Principal component analysis of high-frequency data. J Am Stat Assoc 114(525):287–303

    Article  MathSciNet  MATH  Google Scholar 

  2. Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404

    Article  Google Scholar 

  3. Ben-Hur A, Horn D, Siegelmann HT, Vapnik V (2001) Support vector clustering. J Mach Learn Res 2(2):125–137

    MATH  Google Scholar 

  4. Bourgain J, Dirksen S, Nelson J (2015) Toward a unified theory of sparse dimensionality reduction in euclidean space. Geom Funct Anal 25(4):1009–1088

    Article  MathSciNet  MATH  Google Scholar 

  5. Camastra F, Verri A (2005) A novel kernel method for clustering. IEEE Trans Pattern Anal Mach Intell 27(5):801–805

    Article  Google Scholar 

  6. Chen S, Zhao H, Kong M, Luo B (2007) 2d-lpp: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4):912–921

    Article  Google Scholar 

  7. Chiang JH, Hao PY (2003) A new kernel-based fuzzy clustering approach: support vector clustering with cell growing. IEEE Trans Fuzzy Syst 11(4):518–527

    Article  Google Scholar 

  8. Dhanachandra N, Manglem K, Chanu YJ (2015) Image segmentation using k -means clustering algorithm and subtractive clustering algorithm. Proc Comput Sci 54:764–771

    Article  Google Scholar 

  9. Gao L, Li X, Song J, Shen HT (2019) Hierarchical LSTMS with adaptive attention for visual captioning. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2019.2894139

    Article  Google Scholar 

  10. Graves D, Pedrycz W (2010) Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study. Fuzzy Sets Syst 161(4):522–543

    Article  MathSciNet  Google Scholar 

  11. Gui J, Sun Z, Ji S, Tao D, Tan T (2017) Feature selection based on structured sparsity: a comprehensive study. IEEE Trans Neural Netw Learn Syst 28(7):1490–1507

    Article  MathSciNet  Google Scholar 

  12. Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. J Intell Inf Syst 17(2–3):107–145

    Article  MATH  Google Scholar 

  13. Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. J Roy Stat Soc 28(1):100–108

    MATH  Google Scholar 

  14. He X, Niyogi P (2004) Locality preserving projections. In: Thrun S, Saul LK, Schölkopf B (eds) Advances in neural information processing systems. MIT Press, pp. 153–160. http://papers.nips.cc/paper/2359-locality-preserving-projections.pdf

  15. Hu R, Zhu X, Zhu Y, Gan J (2019) Robust SVM with adaptive graph learning. World Wide Web. https://doi.org/10.1007/s11280-019-00766-x

    Article  Google Scholar 

  16. Inokuchi R, Miyamoto S (2004) LVQ clustering and SOM using a kernel function. In: 2004 IEEE international conference on fuzzy systems (IEEE cat. no. 04CH37542), vol 3. IEEE, pp 1497–1500

  17. Jain AK, Dubes RC et al (1988) Algorithms for clustering data, vol 6. Prentice Hall, Englewood Cliffs

    MATH  Google Scholar 

  18. Ji Y, Zhan Y, Yang Y, Xu X, Shen F, Shen HT (2019) A knowledge map guided coarse-to-fine action recognition. IEEE Trans Image Process. https://doi.org/10.1109/TIP.2019.2952088

    Article  Google Scholar 

  19. Kalousis A, Prados J, Hilario M (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12(1):95–116

    Article  Google Scholar 

  20. Kannan SR, Sathya A, Sathya A, Sathya A (2011) Robust kernel FCM in segmentation of breast medical images. Expert Syst Appl Int J 38(4):4382–4389

    Article  Google Scholar 

  21. Lei Z, Zhu S (2007) Face recognition based on orthogonal discriminant locality preserving projections. Neurocomputing 70(7):1543–1546

    Google Scholar 

  22. Li Z, Zhang Z, Qin J, Zhang Z, Shao L (2019) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst 99(1):1–14

    Google Scholar 

  23. Liu Y, Jiao L, Shang F (2013) An efficient matrix factorization based low-rank representation for subspace clustering. Pattern Recogn 46(1):284–292

    Article  MATH  Google Scholar 

  24. Luxburg UV (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  25. Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181

    Article  Google Scholar 

  26. Nesterov Y (2018) Lectures on convex optimization, vol 137. Springer, Berlin

    Book  MATH  Google Scholar 

  27. Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Proc Nips 14:849–856

    Google Scholar 

  28. Nie F, Rui Z, Xuelong LI (2017) A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci China (Information Sciences) 60(11):112101

    Article  MathSciNet  Google Scholar 

  29. Nie F, Xu D, Tsang IW, Zhang C (2009) Spectral embedded clustering. In: International joint conference on artificial intelligence

  30. Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565

    Article  Google Scholar 

  31. Patel VM, Van Nguyen H, Vidal R (2013) Latent space sparse subspace clustering. In: Proceedings of the IEEE international conference on computer vision, pp 225–232

  32. Ren X, Elkishky A, Wang C, Tao F, Voss CR, Ji H, Han J (2015) Clustype: effective entity recognition and typing by relation phrase-based clustering. In: Kdd 2015, pp 995–1004

  33. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  34. Sánchez A VD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20

    Article  Google Scholar 

  35. Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: International conference on artificial neural networks. Springer, pp 175–184

  36. Schlittgen R (2011) A weighted least-squares approach to clusterwise regression. Asta Adv Stat Anal 95(2):205–217

    Article  MathSciNet  MATH  Google Scholar 

  37. Schölkopf B, Smola A, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319

    Article  Google Scholar 

  38. Wang B, Yang Y, Xu X, Hanjalic A, Shen HT (2017) Adversarial cross-modal retrieval. In: Proceedings of the 2017 ACM on multimedia conference, pp 154–162

  39. Wang S, Nie F, Chang X, Li X, Sheng QZ, Yao L (2016) Uncovering locally discriminative structure for feature analysis. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 281–295

  40. Wang W, Liu B, Guan H, Zhou Y, Xia S (2015) Spectral embedded clustering algorithm based on kernel function. J Comput Appl 35(3):761–765,810

    Google Scholar 

  41. Wen G (2019) Robust self-tuning spectral clustering. Neurocomputing. https://doi.org/10.1016/j.neucom.2018.11.105

    Article  Google Scholar 

  42. Xie GS, Zhang Z, Liu L, Zhu F, Zhang XY, Shao L, Li X (2019) SRSC: selective, robust, and supervised constrained feature representation for image classification. IEEE Trans Neural Netw Learn Syst 99(1):1–14

    Google Scholar 

  43. Zhan M, Lu G, Wen G, Zhang L, Wu L (2019) A clustering algorithm via kernel function and locality preserving projections. In: IEEE symposium series on computational intelligence

  44. Zhang DQ, Chen SC (2003) Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Process Lett 18(3):155–162

    Article  Google Scholar 

  45. Zhang S, Wong HS (2010) ARIMP: a generalized adjusted rand index for cluster ensembles. In: 2010 20th international conference on pattern recognition. IEEE, pp 778–781

  46. Zhang Z, Lai Z, Huang Z, Wong W, Xie G, Liu L, Shao L (2019) Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Trans Image Process 28(10):4803–4818

    Article  MathSciNet  MATH  Google Scholar 

  47. Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782

    Article  Google Scholar 

  48. Zheng Q, Liu Z (2016) Research on improved normalized cut spectral clustering algorithm. In: 2016 Chinese control and decision conference (CCDC). IEEE, pp 1981–1984

  49. Zheng W, Zhu X, Zhu Y, Hu R, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimed. Tools Appl. 77(22):29739–29755

    Article  Google Scholar 

  50. Zhu X, Gan J, Lu G, Li J, Zhang S (2019) Spectral clustering via half-quadratic optimization. World Wide Web. https://doi.org/10.1007/s11280-019-00731-8

    Article  Google Scholar 

  51. Zhu X, Li X, Zhang S, Xu Z, Yu L, Wang C (2017) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044

    Article  Google Scholar 

  52. Zhu X, Yang J, Zhang C, Zhang S (2019) Efficient utilization of missing data in cost-sensitive learning. IEEE Trans Knowl Data Eng. https://doi.org/10.1109/TKDE.2019.2956530

    Article  Google Scholar 

  53. Zhu X, Zhang S, Hu R, He W, Lei C, Zhu P (2019) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034

    Article  Google Scholar 

  54. Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2019) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng 31(8):1532–1543

    Article  Google Scholar 

  55. Zhu X, Zhu Y, Zheng W (2019) Spectral rotation for deep one-step clustering. Pattern Recognit. https://doi.org/10.1016/j.patcog.2019.107175

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the China Key Research Program (Grant No: 2016YF-B1000905); the Key Program of the National Natural Science Foundation of China (Grant No: 61836016); the Natural Science Foundation of China (Grants No: 61876046, 61573270, 81701780 and 61672177); the Project of Guangxi Science and Technology (GuiKeAD17195062); the Guangxi Natural Science Foundation (Grant No: 2017GXNSFBA198221); the Guangxi Collaborative Innovation Center of Multi-Source Information Integration and Intelligent Processing; the Guangxi High Institutions Program of Introducing 100 High-Level Overseas Talents; and the Research Fund of Guangxi Key Lab of Multisource Information Mining & Security (18-A-01-01); Innovation Project of Guangxi Graduate Education (Grants No: YCSW2020108, JXXYYJSCXXM-008).

Author information

Authors and Affiliations

  1. Guangxi Key Lab of Multi-Source Information Mining and Security, Guangxi Normal University, Guilin, Guangxi, China

    Mengmeng Zhan, Guangquan Lu, Guoqiu Wen, Leyuan Zhang & Lin Wu

Authors
  1. Mengmeng Zhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guangquan Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Guoqiu Wen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Leyuan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guangquan Lu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhan, M., Lu, G., Wen, G. et al. Using Locality Preserving Projections to Improve the Performance of Kernel Clustering. Neural Process Lett 52, 1827–1842 (2020). https://doi.org/10.1007/s11063-020-10252-5

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-020-10252-5

Keywords

Search

Navigation