Skip to main content
Springer Nature Link
Log in

Bayesian Low-Rank and Sparse Nonlinear Representation for Manifold Clustering

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Linear representation usually used by the optimization model about low-rankness and sparsity limits their applications to some extent. In this paper, we propose Bayesian low-rank and sparse nonlinear representation (BLSN) model exploiting nonlinear representation. Different from the optimization model, BLSN can be solved by traditional algorithm in Bayesian statistics easily without knowing the explicit mapping by kernel trick. Moreover, it can learn the parameters adaptively to choose the low-rank and sparse properties and also provides a way to enforce more properties on one quantity in a Bayesian model. Based on the observation that the data points drawn from a union of manifolds may gain more meaningful linear structure after a nonlinear mapping, we apply BLSN for manifold clustering. It can handle different problems by constructing various kernels. With respect to the case of linear manifold, known as subspace segmentation, we propose a kernel by the Veronese mapping. In addition, we also design the kernel matrices for the case of nonlinear manifold. Experimental results confirm the effectiveness and the potential of our model for manifold clustering.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

Notes

  1. http://sipi.usc.edu/database/database.cgi?volume=textures.

References

  1. Armagan A, Dunson DB, Lee J (2013) Generalized double Pareto shrinkage. Stat Sin 23(1):119–143

    MathSciNet  MATH  Google Scholar 

  2. Babacan SD, Luessi M, Molina R, Katsaggelos AK (2012) Sparse bayesian methods for low-rank matrix estimation. IEEE Trans Signal Process 60(8):3964–3977

    Article  MathSciNet  Google Scholar 

  3. Basri R, Jacobs DW (2003) Lambertian reflectance and linear subspaces. IEEE Trans Pattern Anal Mach Intell 25(2):218–233

    Article  Google Scholar 

  4. Beal M (2003) Variational algorithm for approximate bayesian inference. Ph.D. thesis, University College London

  5. Bishop CM (2006) Pattern recognition and machine learning. Springer, New York

    MATH  Google Scholar 

  6. Carvalho CM, Polson NG (2010) The horseshoe estimator for sparse signals. Biometrika 97(2):465–480

    Article  MathSciNet  MATH  Google Scholar 

  7. Chen G, Atev S, Lerman G (2009) Kernel spectral curvature clustering. CoRR arXiv:0909.1605

  8. Chen G, Lerman G (2009) Spectral curvature clustering. Int J Comput Vis 81(3):317–330

    Article  Google Scholar 

  9. Cheng B, Liu G, Wang J, Huang Z, Yan S (2011) Multi-task low-rank affinity pursuit for image segmentation. In: ICCV, pp 2439–2446

  10. Chhikara RS, Folks JL (1989) The inverse Gaussian distribution: theory, methodology, and applications. Marcel Dekker Inc, New York

    MATH  Google Scholar 

  11. Ding X, He L, Carin L (2011) Bayesian robust principal component analysis. IEEE Trans Image Process 20(12):3419–3430

    Article  MathSciNet  Google Scholar 

  12. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: CVPR, pp 2790–2797

  13. Elhamifar E, Vidal R (2010) Clustering disjoint subspaces via sparse representation. In: ICASSP, pp 1926–1929

  14. Elhamifar E, Vidal R (2013) Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans Pattern Anal Mach Intell 35(11):2765–2781

    Article  Google Scholar 

  15. Feng J, Lin Z, Xu H, Yan S (2014) Robust subspace segmentation with block-diagonal prior. In: CVPR, pp 3818–3825

  16. Figueiredo MAT (2003) Adaptive sparseness for supervised learning. IEEE Trans Pattern Anal Mach Intell 25(9):1150–1159

    Article  MathSciNet  Google Scholar 

  17. Hans C (2009) Bayesian lasso regression. Biometrika 96(4):835–845

    Article  MathSciNet  MATH  Google Scholar 

  18. Harris J (1992) Algebraic geometry. Springer, New York

    Book  MATH  Google Scholar 

  19. Ho J, Yang MH, Lim J, Lee KC, Kriegman DJ (2003) Clustering appearances of objects under varying illumination conditions. In: CVPR, pp 11–18

  20. Hu H, Lin Z, Feng J, Zhou J (2014) Smooth representation clustering. In: CVPR, pp 3834–3841

  21. Jim Philip (2010) Inference with normal-gamma prior distributions in regression problems. Bayesian Anal 5(1):171–188

    Article  MathSciNet  MATH  Google Scholar 

  22. Lang C, Liu G, Yu J, Yan S (2012) Saliency detection by multitask sparsity pursuit. IEEE Trans Image Process 21(3):1327–1338

    Article  MathSciNet  Google Scholar 

  23. Lee KC, Ho J, Kriegman DJ (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698

    Article  Google Scholar 

  24. Lin Z, Chen M, Wu L, Ma Y (2009) The augmented lagrange multiplier method for exact recovery of corrupted low-rank matrices. UIUC Technical Report, UILU-ENG-09-2215

  25. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. IEEE Trans Pattern Anal Mach Intell 35(1):171–184

    Article  Google Scholar 

  26. Liu G, Lin Z, Yu Y (2010) Robust subspace segmentation by low-rank representation. In: ICML, pp 663–670

  27. Liu G, Yan S (2011) Latent low-rank representation for subspace segmentation and feature extraction. In: ICCV, pp 1615–1622

  28. Liu R, Lin Z, la Torre FD, Su Z (2012) Fixed-rank representation for unsupervised visual learning. In: CVPR, pp 598–605

  29. Lowe DG (1999) Object recognition from local scale-invariant features. In: ICCV, pp 1150–1157

  30. Lu CY, Feng J, Lin Z, Yan S (2013) Correlation adaptive subspace segmentation by trace lasso. In: ICCV, pp 1345–1352

  31. Lu CY, Min H, Zhao ZQ, Zhu L, Huang DS, Yan S (2012) Robust and efficient subspace segmentation via least squares regression. In: ECCV, pp 347–360

  32. von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  33. Ojala T, Pietikäinen M, Mäenpää T (2000) Gray scale and rotation invariant texture classification with local binary patterns. In: ECCV, pp 404–420

  34. Patel VM, Vidal R (2014) Kernel sparse subspace clustering. In: ICIP

  35. Rao S, Tron R, Vidal R, Ma Y (2010) Motion segmentation in the presence of outlying, incomplete, or corrupted trajectories. IEEE Trans Pattern Anal Mach Intell 32(10):1832–1845

    Article  Google Scholar 

  36. Shawe-Taylor J, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge university press, Cambridge

    Book  MATH  Google Scholar 

  37. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  38. Soltanolkotabi M, Candès EJ (2011) A geometric analysis of subspace clustering with outliers. CoRR arXiv:1112.4258

  39. Souvenir R, Pless R (2005) Manifold clustering. In: ICCV, pp 648–653

  40. Tang K, Liu R, Su Z, Zhang J (2014) Structure-constrained low-rank representation. IEEE Trans Neural Netw Learn Syst 25:2167–2179

    Article  Google Scholar 

  41. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc 58(1):267–288

    MathSciNet  MATH  Google Scholar 

  42. Tipping ME (2001) Sparse bayesian learning and the relevance vector machine. J Mach Learn Res 1:211–244

    MathSciNet  MATH  Google Scholar 

  43. Tipping ME, Bishop CM (1999) Mixtures of probabilistic principal component analysers. Neural Comput 11(2):443–482

    Article  Google Scholar 

  44. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: CVPR

  45. Vidal R (2011) Subspace clustering. IEEE Signal Process Mag 28(2):52–68

    Article  Google Scholar 

  46. Wang JJ, Bensmail H, Gao X (2014) Feature selection and multi-kernel learning for sparse representation on a manifold. Neural Netw 51:9–16

    Article  MATH  Google Scholar 

  47. Wang Y, Jiang Y, Wu Y, Zhou Z (2010) Multi-manifold clustering. In: PRICAI, pp 280–291

  48. Wang Y, Jiang Y, Wu Y, Zhou Z (2011) Spectral clustering on multiple manifolds. IEEE Trans Neural Netw 22(7):1149–1161

    Article  Google Scholar 

  49. Yan J, Pollefeys M (2006) A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: ECCV, pp 94–106

  50. Yang AY, Wright J, Ma Y, Sastry SS (2008) Unsupervised segmentation of natural images via lossy data compression. Comput Vis Image Underst 110(2):212–225

    Article  Google Scholar 

  51. Zhang T, Ghanem B, Liu S, Ahuja N (2012) Low-rank sparse learning for robust visual tracking. In: ECCV, pp 470–484

  52. Zhang X (2004) Matrix analysis and applications. Springer, New York

    Google Scholar 

  53. Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: CVPR, pp 2328–2335

Download references

Acknowledgments

The work of Z. Su was supported by the National Nature Science Foundation of China (No. 61572099, 61173103, 91230103), National Science and Technology Major Project (No. 2013ZX04005021, 2014ZX04001011).

Author information

Authors and Affiliations

  1. School of Mathematics, Liaoning Normal University, No. 850 Huanghe Road, Shahekou District, Dalian, 116029, Liaoning, People’s Republic of China

    Kewei Tang & Jie Zhang

  2. School of Mathematical Sciences, Dalian University of Technology, No. 2 Linggong Road, Ganjingzi District, Dalian, 116024, Liaoning, People’s Republic of China

    Zhixun Su & Jiangxin Dong

Authors
  1. Kewei Tang
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Jie Zhang
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Zhixun Su
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Jiangxin Dong
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding authors

Correspondence to Kewei Tang or Zhixun Su.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, K., Zhang, J., Su, Z. et al. Bayesian Low-Rank and Sparse Nonlinear Representation for Manifold Clustering. Neural Process Lett 44, 719–733 (2016). https://doi.org/10.1007/s11063-015-9490-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-015-9490-x

Keywords