Skip to main content
SpringerLink
Log in

Dual subspace learning via geodesic search on Stiefel manifold

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

Oja’s principal subspace algorithm is a well-known and powerful technique for learning and tracking principal information of time series. However, Oja’s algorithm is divergent when performing the task of minor subspace analysis. In the present paper, we transform Oja’s algorithm into a dual learning algorithm in the sense of fulfilling principal subspace analysis as well as minor subspace analysis via geodesic search on Stiefel manifold. Also inherent stability is guaranteed for the proposed geodesic based algorithm due to the fact the weight matrix rigourously evolves on the compact Stiefel manifold. The effectiveness of the proposed algorithm is further verified in the section of numerical simulation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Comon P, Golub GH (1990) Tracking a few extreme singular values and vectors in signal processing. Proc IEEE 78(8):1327–1343

    Article  Google Scholar 

  2. Li N, Guo G, Chen L, Chen S (2013) Optimal subspace classification method for complex data. Int J Mach Learn Cybern 4(2):163–171

    Article  Google Scholar 

  3. Guo G, Chen S, Chen L (2012) Soft subspace clustering with an improved feature weight self-adjustment mechanism. Int J Mach Learn Cybern 3(1):39–49

    Article  Google Scholar 

  4. Li J, Han G, Wen J, Gao X (2011) Robust tensor subspace learning for anomaly detection. Int J Mach Learn Cybern 2(2):89–98

    Article  Google Scholar 

  5. Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, Baltimore

    MATH  Google Scholar 

  6. Juang BH, Kung SY, Kamm CA (1991) In: Proceedings of the 1991 IEEE workshop neural networks for signal processing. Princeton

  7. Oja E (1992) Principal components, minor components, and linear neural networks. Neural Netw 5(6):927–935

    Article  Google Scholar 

  8. Kung S-Y, Diamantaras KI, Taur J-S (1994) Adaptive principal component extraction (APEX) and applications. IEEE Trans Signal Process 42(5):1202–1217

    Article  Google Scholar 

  9. Xu L (1993) Least mean square error reconstruction principle for self-organizing neural-nets. Neural Netw 6(5):627–648

    Article  Google Scholar 

  10. Chen T, Amari S-I, Lin Q (1998) A unified algorithm for principal and minor components extraction. Neural Netw 11:385–390

    Article  Google Scholar 

  11. Manton JH, Helmke U, Mareels IM (2005) A dual purpose principal and minor component flow. Syst Control Lett 54:759–769

    Article  MATH  MathSciNet  Google Scholar 

  12. Chen T, Amari S-i (2001) Unified stabilization approach to principal and minor components extraction algorithms. Neural Netw 14:1377–1387

    Article  Google Scholar 

  13. Fiori S (2002) A minor subspace algorithm based on neural Stiefel dynamics. Int J Neural Syst 12(5):1–18

    Google Scholar 

  14. Kong X, Hu C, Han C (2012) A dual purpose principal and minor subspace gradient flow. IEEE Trans Signal Process 60(1):197–210

    Article  MathSciNet  Google Scholar 

  15. Abed-Meraim K, Attallah S, Chkeif A, Hua Y (2000) Orthogonal Oja algorithm. IEEE Signal Process Lett 7(5):116–119

    Article  Google Scholar 

  16. Helmke U, Moore JB (1993) Optimization and dynamical systems. Springer, Berlin, Heidelberg

    Google Scholar 

  17. Yan W-Y, Helmke U, Moore JB (1994) Global analysis of oja’s flow for neural networks. IEEE Trans Neural Netw 5(5):674–683

    Article  Google Scholar 

  18. Douglas SC, Kung S, Amari S-i (1998) A self-stabilized minor subspace rule. IEEE Signal Process Lett 5(12):328–330

    Article  Google Scholar 

  19. Absil P-A, Mahony R, Sepulchre R (2009) Optimization algorithms on matrix manifolds. Princeton University Press, Princeton

    Google Scholar 

  20. Yoshizawa S, Helmke U, Starkov K (2001) Convergence analysis for principal component flows. Appl Math Comut Sci 11(1):223–236

    MathSciNet  Google Scholar 

  21. Baldi P, Hornik K (1991) Backpropagation and unsupervised learning in linear networks. In: Chauvin Y, Rumelhart DE (eds) Backpropagation: theory, architectures and applications. Erlbaum Associates, NJ

    Google Scholar 

  22. Edelman A, Arias T, Smith S (1998) The geometry of algorithms with orthogonality constraints. SIAM J Matrix Anal Appl 20(2):303–353

    Article  MATH  MathSciNet  Google Scholar 

  23. Nishimori Y, Akaho S (2005) Learning algorithms utilizing quasi-geodesic flows on the Stiefel manifold. Neurocomputing 67:106–135

    Article  Google Scholar 

  24. Abrudan TE, Eriksson J, Koivunen V (2008) Steepest descent algorithms for optimization under unitary matrix constraint. IEEE Trans Signal Process 56(3):1134–1147

    Article  MathSciNet  Google Scholar 

  25. Webers C, Manton JH (2006) A generalisation of the oja subspace flow. 17th international symposium on mathematical theory of networks and systems, pp 24–28

Download references

Acknowledgments

The authors would like to acknowledge the reviewers for their helpful comments and suggestions for improvements of this paper. Supported by the National Natural Science Foundation of China under Grant No.61002039 and No.61202254; The Fundamental Research Funds for the Central Universities under Grant No. DC12010216 and No. 0913130475.

Author information

Authors and Affiliations

  1. School of Science, Dalian Nationalities University, Dalian, 116600, China

    Lijun Liu & Rendong Ge

  2. College of Computer Science and Technology, Dalian Nationalities University, Dalian, 116600, China

    Jiana Meng

  3. College of Foreign Languages and Cultures, Dalian Nationalities University, Dalian, 116600, China

    Guangjie You

Authors
  1. Lijun Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rendong Ge
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jiana Meng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Guangjie You
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lijun Liu.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, L., Ge, R., Meng, J. et al. Dual subspace learning via geodesic search on Stiefel manifold. Int. J. Mach. Learn. & Cyber. 5, 753–759 (2014). https://doi.org/10.1007/s13042-013-0217-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-013-0217-x

Keywords

Search

Navigation