Skip to main content
SpringerLink
Log in

Geometric weighting subspace clustering on nonlinear manifolds

  • 1135T: Social Multimedia Processing
  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Considering that the conventional subspace clustering methods of sparse subspace clustering (SSC) and low-rank representation (LRR) are only applicable to linear manifolds, we propose a novel subspace clustering framework that generalizes them for nonlinear manifolds. To do this, we integrate a weighting matrix and kernel matrix into the regularization of this framework. The weighting matrix is calculated using the similarities between tangent spaces on data manifolds and the Euclidean distances between data points, so that it can explicitly characterize the intrinsic geometry of data manifolds. Besides, we provide a geometrical interpretation for the effects of weighted ℓ∗-norm involved in the proposed framework, exploiting symmetric gauge function (SGF) of von Neumann theory that establishes a relationship exactly between singular and matrix norm. To solve the regularization with respect to the weighted norm, we design a fixed-point continuation algorithm to obtain an approximate closed solution. Experimental results on three computer vision tasks show the superiority of clustering accuracy over other similar approaches and demonstrate the effectiveness of the weighting matrix. That also proves the proposed method has better interpretability than other state-of-the-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Bai L, Liang J (2020) Sparse subspace clustering with entropy-norm. In: International conference on machine learning, pp 561–568. PMLR

  2. Basri R, Jacobs DW (2003) Lambertian reflectance and linear subspaces. IEEE Trans Pattern Anal Mach Intell 2:218–233

    Article  Google Scholar 

  3. Bishop CM (2006) Pattern recognition. Mach Learn, 128(9)

  4. Boyd S, Parikh N, Chu E, Peleato B, Eckstein J, et al. (2011) Distributed optimization and statistical learning via the alternating direction method of multipliers. Found and Trends®;, Mach Learn 3 (1):1–122

    MATH  Google Scholar 

  5. Cai JF, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982

    Article  MathSciNet  MATH  Google Scholar 

  6. Candes EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted l1 minimization. J Fourier Anal Applic 14(5–6):877–905

    Article  MATH  Google Scholar 

  7. Chen Y, Li CG, You C (2020) Stochastic sparse subspace clustering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 4155–4164

  8. Chen Y, Li G, Gu Y (2017) Active orthogonal matching pursuit for sparse subspace clustering. IEEE Signal Process Lett 25(2):164–168

    Article  Google Scholar 

  9. Chun-Guang Li, Chong Y, René V (2017) Structured sparse subspace clustering: a joint affinity learning and subspace clustering framework. IEEE Transactions on Image Processing

  10. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: 2009 IEEE Conference on computer vision and pattern recognition, pp 2790–2797. IEEE

  11. Elhamifar E, Vidal R (2013) Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Article  Google Scholar 

  12. Goh A, Vidal R (2008) Clustering and dimensionality reduction on riemannian manifolds. In: 2008 IEEE Conference on computer vision and pattern recognition, pp 1–7. IEEE

  13. Goldfarb D, Ma S (2011) Convergence of fixed-point continuation algorithms for matrix rank minimization. Found Comput Math 11(2):183–210

    Article  MathSciNet  MATH  Google Scholar 

  14. Gu S, Zhang L, Zuo W, Feng X (2014) Weighted nuclear norm minimization with application to image denoising. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2862–2869

  15. Hale ET, Yin W, Zhang Y (2008) Fixed-point continuation for l1-minimization: methodology and convergence. SIAM J Optim 19(3):1107–1130

    Article  MathSciNet  MATH  Google Scholar 

  16. Harandi MT, Hartley R, Lovell B, Sanderson C (2015) Sparse coding on symmetric positive definite manifolds using Bregman divergences. IEEE Trans Neural Netw Learn Syst 27(6):1294–1306

    Article  Google Scholar 

  17. Ji P, Zhang T, Li H, Salzmann M, Reid I (2017) Deep subspace clustering networks. In: Neural information processing systems (NIPS), CONF

  18. Kheirandishfard M, Zohrizadeh F, Kamangar F (2020) Deep low-rank subspace clustering. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 864–865

  19. Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 5:684–698

    Google Scholar 

  20. Lewis AS (2003) The mathematics of eigenvalue optimization. Math Program 97(1–2):155–176

    Article  MathSciNet  MATH  Google Scholar 

  21. Lin Z, Chen M, Ma Y (2010) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices, arXiv:1009.50551009.5055

  22. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2012) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  23. Liu J, Han J (2018) Spectral clustering. In: Data clustering, pp 177–200. Chapman and Hall/CRC

  24. Lu C, Feng J, Lin Z, Mei T, Yan S (2018) Subspace clustering by block diagonal representation. IEEE Trans Pattern Anal Machine Intell 41(2):487–501

    Article  Google Scholar 

  25. Nguyen H, Yang W, Shen F, Sun C (2015) Kernel low-rank representation for face recognition. Neurocomputing 155:32–42

    Article  Google Scholar 

  26. Patel VM, Van Nguyen H, Vidal R (2013) Latent space sparse subspace clustering. In: Proceedings of the IEEE international conference on computer vision, pp 225–232

  27. Patel VM, Vidal R (2014) Kernel sparse subspace clustering. In: 2014 IEEE international conference on image processing (ICIP), pp 2849–2853. IEEE

  28. Peng X, Feng J, Zhou J, Lei Y, Yan S (2020) Deep subspace clustering. IEEE Trans Neural Netw Learn Syst 31(12):5509–5521

    Article  MathSciNet  Google Scholar 

  29. Pham DS, Budhaditya S, Phung D, Venkatesh S (2012) Improved subspace clustering via exploitation of spatial constraints. In: 2012 IEEE Conference on computer vision and pattern recognition, pp 550–557. IEEE

  30. Roweis ST, Saul LK (2000) . Nonlinear dimensionality reduction by locally linear embedding science 290(5500):2323–2326

    Google Scholar 

  31. Soltanolkotabi M, Candes EJ, et al. (2012) A geometric analysis of subspace clustering with outliers. Ann Stat 40(4):2195–2238

    Article  MathSciNet  MATH  Google Scholar 

  32. Somandepalli K, Narayanan S (2019) Reinforcing self-expressive representation with constraint propagation for face clustering in movies. In: ICASSP 2019-2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 4065–4069. IEEE

  33. Tenenbaum JB, De Silva V, Langford JC (2000) . A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323

    Google Scholar 

  34. Tomasi C, Kanade T (1992) Shape and motion from image streams under orthography: a factorization method. Int J Comput Vis 9(2):137–154

    Article  Google Scholar 

  35. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: 2007 IEEE conference on computer vision and pattern recognition, pp 1–8. IEEE

  36. Vidal R (2011) Subspace clustering. IEEE Signal Process Mag 28 (2):52–68

    Article  Google Scholar 

  37. Vidal R, Favaro P (2014) Low rank subspace clustering (lrsc). Pattern Recogn Lett 43:47–61

    Article  Google Scholar 

  38. Von Neumann J (1937) Some matrix-inequalities and metrization of matric space

  39. Wang P, Han B, Li J, Gao X (2019) Structural reweight sparse subspace clustering. Neural Process Lett 49(3):965–977

    Article  Google Scholar 

  40. Wang Y, Jiang Y, Wu Y, Zhou ZH (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161

    Article  Google Scholar 

  41. Wang Y, Wang YX, Singh A (2018) A theoretical analysis of noisy sparse subspace clustering on dimensionality-reduced data. IEEE Trans Inf Theory 65(2):685–706

    Article  MathSciNet  MATH  Google Scholar 

  42. Watson GA (1992) Characterization of the subdifferential of some matrix norms. Linear Algebra Applic 170:33–45

    Article  MathSciNet  MATH  Google Scholar 

  43. Wu JY, Huang LC, Yang MH, Chang LH, Liu CH (2019) Sparse subspace clustering with sequentially ordered and weighted l1-minimization. In: 2019 IEEE International conference on image processing (ICIP), pp 3387–3391. IEEE

  44. Xia G, Sun H, Feng L, Zhang G, Liu Y (2017) Human motion segmentation via robust kernel sparse subspace clustering. IEEE Trans Image Process 27(1):135–150

    Article  MathSciNet  MATH  Google Scholar 

  45. Xiao S, Tan M, Xu D, Dong ZY (2015) Robust kernel low-rank representation. IEEE Trans Neur Netw Learn Syst 27(11):2268–2281

    Article  MathSciNet  Google Scholar 

  46. Yang C, Ren Z, Sun Q, Wu M, Yin M, Sun Y (2019) Joint correntropy metric weighting and block diagonal regularizer for robust multiple kernel subspace clustering. Inform Sci 500:48–66

    Article  MathSciNet  MATH  Google Scholar 

  47. Yin M, Guo Y, Gao J, He Z, Xie S (2016) Kernel sparse subspace clustering on symmetric positive definite manifolds. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 5157–5164

  48. You C, Robinson D, Vidal R (2016) Scalable sparse subspace clustering by orthogonal matching pursuit. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3918–3927

  49. You C, Vidal R (2015) Geometric conditions for subspace-sparse recovery. In: International conference on machine learning, pp 1585–1593. PMLR

  50. Zhai H, Zhang H, Zhang L, Li P, Plaza A (2016) A new sparse subspace clustering algorithm for hyperspectral remote sensing imagery. IEEE Geosci Remote Sens Lett 14(1):43–47

    Article  Google Scholar 

  51. Zhan J, Zhu Y, Bai Z (2020) An expression-reinforced sparse subspace clustering by orthogonal matching pursuit. In: 2020 IEEE International conference on image processing (ICIP), pp 211–215. IEEE

  52. Zhang J, Li CG, You C, Qi X, Zhang H, Guo J, Lin Z (2019) Self-supervised convolutional subspace clustering network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 5473–5482

  53. Zhang Q, Wu YN, Zhu SC (2017) Interpretable convolutional neural networks. In: 2018 IEEE/CVF conference on computer vision and pattern recognition

  54. Zhang T, Yang J, Zhao D, Ge X (2007) Linear local tangent space alignment and application to face recognition. Neurocomputing 70(7-9):1547–1553

    Article  Google Scholar 

  55. Zhou L, Xiao B, Liu X, Zhou J, Hancock ER, et al. (2019) Latent distribution preserving deep subspace clustering. In: 28th International joint conference on artificial intelligence, pp 4440–4446. York

  56. Zhou P, Hou Y, Feng J (2018) Deep adversarial subspace clustering. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 1596–1604

  57. Zhu W, Peng B (2020) Sparse and low-rank regularized deep subspace clustering. Knowl-Based Syst 204:106199

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Geophysics, Chengdu University of Technology, Chengdu, 610059, Sichuan, China

    Shujun Liu & Huajun Wang

Authors
  1. Shujun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huajun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shujun Liu.

Ethics declarations

Conflict of Interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Liu, S., Wang, H. Geometric weighting subspace clustering on nonlinear manifolds. Multimed Tools Appl 81, 42971–42990 (2022). https://doi.org/10.1007/s11042-022-12797-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12797-0

Keywords

Search

Navigation