Skip to main content
SpringerLink
Log in

Constructing \(L_{1}\)-graphs for subspace learning via recurrent neural networks

  • Theoretical Advances
  • Published:
Pattern Analysis and Applications Aims and scope Submit manuscript

Abstract

In this paper, the problem that we are interested is constructing \(l_{1}\)-graphs for subspace learning via recurrent neural networks. We propose a closed affine subspace learning (CASL) method to do so. The problem of CASL is formulated as an optimization problem described by an energy function in the non-negative orthant. The sufficient conditions for keeping the minimum points of the energy function in the non-negative orthant are given. A model of Lotka–Volterra recurrent neural networks is constructed to solve the optimization problem in terms of these sufficient conditions. It shows that the set of stable attractors of the network model just equals the set of minimum points of the energy function in the non-negative orthant. Based on these equivalences, the problem of CASL can be solved by running the proposed LV RNNs to obtain the stable attractors. The \(l_{1}\)-graphs then can be constructed. Experiments on some synthetic and real data show that the \(l_{1}\)-graphs constructed in this way are more effective, especially when processing the data sampled from multiple manifolds.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Asai T, Ohtani M, Yonezu H (1999) Analog integrated circuits for the lotka-volterra competitive neural networks. IEEE Trans Neural Netw 10(5):1222–1231

    Article  Google Scholar 

  2. Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720

  3. Belkin M, Matveeva I, Niyogi P (2004) Regularization and semi-supervised learning on large graphs. In: Learning theory. Springer, Berlin, pp 624–638

  4. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MATH  MathSciNet  Google Scholar 

  5. Brox T, Malik J (2010) Object segmentation by long term analysis of point trajectories. In: Computer vision-ECCV 2010. Springer, Berlin, pp 282–295

  6. Chen G, Lerman G (2009) Spectral curvature clustering (scc). Int J Comput Vis 81(3):317–330

    Article  Google Scholar 

  7. Cheng B, Yang J, Yan S, Fu Y, Huang TS (2010) Learning with l 1-graph for image analysis. IEEE Trans Image Process 19(4):858–866

    Article  MathSciNet  Google Scholar 

  8. Donoho DL, Grimes C (2003) Hessian eigenmaps: locally linear embedding techniques for high-dimensional data. Proc Natl Acad Sci 100(10):5591–5596

    Article  MATH  MathSciNet  Google Scholar 

  9. Elhamifar E, Vidal R (2009) Sparse subspace clustering. In: IEEE conference on computer vision and pattern recognition, 2009. CVPR 2009. IEEE, pp 2790–2797

  10. Elhamifar E, Vidal R (2011) Sparse manifold clustering and embedding. In: Advances in neural information processing systems, pp 55–63

  11. Fukai T, Tanaka S (1997) A simple neural network exhibiting selective activation of neuronal ensembles: from winner-take-all to winners-share-all. Neural Comput 9(1):77–97

    Article  MATH  Google Scholar 

  12. Ge SS, Guan F, Loh A, Fua CH (2006) Feature representation based on intrinsic structure discovery in high dimensional space. In: Robotics and Automation, 2006. ICRA 2006. IEEE Conference on Proceedings 2006. IEEE, pp 3399–3404

  13. Grant M, Boyd S, Ye Y (2008) Cvx: Matlab software for disciplined convex programming

  14. Guan N, Tao D, Luo Z, Yuan B (2011a) Manifold regularized discriminative nonnegative matrix factorization with fast gradient descent. IEEE Trans Image Process 20(7):2030–2048

    Article  MathSciNet  Google Scholar 

  15. Guan N, Tao D, Luo Z, Yuan B (2011b) Non-negative patch alignment framework. IEEE Trans Neural Netw 22(8):1218–1230

    Article  Google Scholar 

  16. Guan N, Tao D, Luo Z, Shawe-Taylor J (2012a) Mahnmf: Manhattan non-negative matrix factorization. arXiv, preprint arXiv:12073438

  17. Guan N, Tao D, Luo Z, Yuan B (2012b) Nenmf: an optimal gradient method for nonnegative matrix factorization. IEEE Trans Signal Process 60(6):2882–2898

    Article  MathSciNet  Google Scholar 

  18. Lauer F, Schnorr C (2009) Spectral clustering of linear subspaces for motion segmentation. In: IEEE transactions on computer vision. IEEE, pp 678–685

  19. Lei YK, Xu YM, Yang JA, Ding ZG, Gui J (2012) Feature extraction using orthogonal discriminant local tangent space alignment. Pattern Anal Appl 15(3):249–259

    Article  MathSciNet  Google Scholar 

  20. Liu G, Yan S (2011) Latent low-rank representation for subspace segmentation and feature extraction. In: 2011 IEEE International Conference on Computer Vision (ICCV). IEEE, pp 1615–1622

  21. Ng AY, Jordan MI, Weiss Y et al (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 2:849–856

    Google Scholar 

  22. Nistér D (2005) Preemptive ransac for live structure and motion estimation. Mach Vis Appl 16(5):321–329

    Article  Google Scholar 

  23. Pan Y, Ge SS (2009) Weighted locally linear embedding for dimension reduction. Pattern Recognit 42(5):798–811

    Article  MATH  Google Scholar 

  24. Rao SR, Tron R, Vidal R, Ma Y (2008) Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8

  25. Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326

    Article  Google Scholar 

  26. Saul LK, Roweis ST (2003) Think globally, fit locally: unsupervised learning of low dimensional manifolds. J Mach Learn Res 4:119–155

    MathSciNet  Google Scholar 

  27. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  28. Short R, Fukunaga K (1980) A new nearest neighbor distance measure. In: Proceedings of the fifth international Conference on Pattern Recognition, pp 81–86

  29. Tenenbaum JB (1998) Mapping a manifold of perceptual observations. In: Advances in neural information processing systems, pp 682–688

  30. Tomasi C, Kanade T (1992) Shape and motion from image streams under orthography: a factorization method. Int J Comput Vis 9(2):137–154

    Article  Google Scholar 

  31. Tron R, Vidal R (2007) A benchmark for the comparison of 3-d motion segmentation algorithms. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8

  32. Turk M, Pentland A (1991) Eigenfaces for recognition. J Cognit Neurosci 3(1):71–86

    Article  Google Scholar 

  33. Van Den Berg E, Friedlander MP (2008) Probing the pareto frontier for basis pursuit solutions. SIAM J Sci Comput 31(2):890–912

    Article  MATH  MathSciNet  Google Scholar 

  34. Vidal R, Hartley R (2004) Motion segmentation with missing data using powerfactorization and gpca. In: Computer Vision and Pattern Recognition, 2004. CVPR 2004. In: IEEE computer society conference on proceedings of the 2004. IEEE, vol 2, pp II-310

  35. Wright J, Yang AY, Ganesh A, Sastry SS, Ma Y (2009) Robust face recognition via sparse representation. IEEE Trans Pattern Anal Mach Intell 31(2):210–227

    Article  Google Scholar 

  36. Xu D, Yan S, Tao D, Lin S, Zhang HJ (2007) Marginal fisher analysis and its variants for human gait recognition and content-based image retrieval. IEEE Trans Image Process 16(11):2811–2821

    Article  MathSciNet  Google Scholar 

  37. Yan J, Pollefeys M (2006) A general framework for motion segmentation: Independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In: Computer vision-ECCV 2006. Springer, Berlin, pp 94–106

  38. Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2007) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51

    Article  Google Scholar 

  39. Yang B, Chen S, Wu X (2011) A structurally motivated framework for discriminant analysis. Pattern Anal Appl 14(4):349–367

    Article  MathSciNet  Google Scholar 

  40. Yi Z (2010) Foundations of implementing the competitive layer model by lotka-volterra recurrent neural networks. IEEE Trans Neural Netw 21(3):494–507

    Article  Google Scholar 

  41. Yi Z, Tan K (2002) Dynamic stability conditions for lotka-volterra recurrent neural networks with delays. Phys Rev E 66(1):011910

  42. Yu J, Amores J, Sebe N, Radeva P, Tian Q (2008) Distance learning for similarity estimation. IEEE Trans Pattern Anal Mach Intell 30(3):451–462

    Article  Google Scholar 

  43. Zhang Y, Tan KK (2003) Convergence analysis of recurrent neural networks. Kluwer academic, Boston

  44. Zhu X, Ghahramani Z, Lafferty J et al (2003) Semi-supervised learning using gaussian fields and harmonic functions. ICML 3:912–919

    Google Scholar 

Download references

Acknowledgments

This work was supported by National Basic Research Program of China (973 Program) under Grant No. 2011CB302201, Program for New Century Excellent Talents in University under Grant No. NCET-11-0355, and Doctoral Fund of Ministry of Education of China under Grants Nos. 20100181120030 and 20110181110049.

Author information

Authors and Affiliations

  1. Machine Intelligence Laboratory, College of Computer Science, Sichuan University, Chengdu, 610065, People’s Republic of China

    Yin Kuang, Lei Zhang & Zhang Yi

  2. College of Computer Science, Chengdu Normal University, Chengdu, 611130, People’s Republic of China

    Yin Kuang

Authors
  1. Yin Kuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhang Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhang.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kuang, Y., Zhang, L. & Yi, Z. Constructing \(L_{1}\)-graphs for subspace learning via recurrent neural networks. Pattern Anal Applic 18, 817–828 (2015). https://doi.org/10.1007/s10044-014-0370-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10044-014-0370-1

Keywords

Search

Navigation