Skip to main content
SpringerLink
Log in

Global Minima Analysis of Lee and Seung’s NMF Algorithms

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Lee and Seung proposed nonnegative matrix factorization (NMF) algorithms to decompose patterns and images for structure retrieving. The NMF algorithms have been applied to various optimization problems. However, it is difficult to prove the convergence of this class of learning algorithms. This paper presents the global minima analysis of the NMF algorithms. In the analysis, invariant set is constructed so that the non-divergence of the algorithms can be guaranteed in the set. Using the features of linear equation systems and their solutions, the fixed points and convergence properties of the update algorithms are discussed in detail. The analysis shows that, although the cost function is not convex in both A and X together, it is possible to obtain the global minima from the particular learning algorithms. For different initializations, simulations are presented to confirm the analysis results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Lee DD, Seung HS (1999) Learning of the parts of objects by non-negative matrix factorization. Nature 401: 788–791

    Article  Google Scholar 

  2. Cichocki A, Zdunek R, Amari S (2006) New algorithms for non-negative matrix factorization in applications to blind source separation, ICASSP-2006, Toulouse, pp 621–625

  3. Cichocki A, Zdunek R, Amari S (2006) Csiszar’s divergences for non-negative matrix factorization: family of new algorithms. In: 6th International conference on independent component analysis and blind signal separation, Springer LNCS 3889, Charleston, pp 32–39

  4. Hoyer PO (2002) Non-negative sparse coding. In: Proceedings of IEEE workshop on neural networks for signal processing, pp 557–565

  5. Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5: 1457–1469

    MathSciNet  MATH  Google Scholar 

  6. Plumbly M, Oja E (2004) A “non-negative PCA” algorithm for independent component analysis. IEEE Trans Neural Netw 15(1):66–76

    Google Scholar 

  7. Oja E, Plumbley M (2004) Blind separation of positive sources by globally covergent graditent search. Neural Comput 16(9): 1811–1925

    Article  MATH  Google Scholar 

  8. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization 13, NIPS. MIT Press, Cambridge

  9. Chu M, Diele F, Plemmons R, Ragni S (2004) Optimality, computation, and interpretation of nonnegative matrix factorizations. Technical report, Wake Forest University, North Carolina

  10. Berry M, Browne M, Langville A, Pauca V, Plemmons R (2007) Algorithms and applications for approximate nonnegative matrix factorization. Comput Stat Data Anal 52(1): 155–173

    Article  MathSciNet  MATH  Google Scholar 

  11. Lin CJ (2007) On the convergence of multiplicative update algorithms for non-negative matrix factorization. IEEE Trans Neural Netw 18(6): 1589–1596

    Article  Google Scholar 

  12. Gonzales EF, Zhang Y (2005) Accelerating the Lee-Seung algorithm for non-negative matrix factorization. Technical report, Department of Computational and Applied Mathematics, Rice University

  13. Finesso L, Spreij P (2006) Nonnegative matrix factorization and I-divergence alternating minimization. Linear Algebra Appl 416(2-3): 270–287

    Article  MathSciNet  MATH  Google Scholar 

  14. Badeau R, Bertin N, Vincent E (2010) Stability analysis of multiplicative update algorithms and application to non-negative matrix factorization. IEEE Trans Neural Netw 21(12): 1869–1881

    Article  Google Scholar 

  15. Yang S, Yi Z (2010) Convergence analysis of non-negative matrix factorization for bss algorithm. Neural Process Lett 31(1): 45–64

    Article  Google Scholar 

  16. Yang S., Ye M. (2012) Multistability of α-divergence based NMF algorithms. Comput Math Appl. 64(2): 73–88

    Article  MathSciNet  MATH  Google Scholar 

  17. Zhou G, Yang Z, Xie S (2011) Online blind source separationusing incremental nonnegative matrix factorization with volume constraint. IEEE Trans Neural Netw 22(4): 550–560

    Article  MathSciNet  Google Scholar 

  18. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8): 1548–1560

    Article  Google Scholar 

  19. Spratling MW (2006) Learning image components for object recognition. J Mach Learn Res 7: 793–815

    MathSciNet  MATH  Google Scholar 

  20. Zhi R, Flierl M, Ruan Q, Kleijn WB (2011) Graph-preserving sparse nonnegative matrix factorization with application to facial expression recognition. IEEE Trans Syst Man Cybern Part B Cybern 41(1): 38–52

    Article  Google Scholar 

  21. Brunet J, Tamayo P, Golub T, Mesirov J (2004) Metagenes and molecular pattern discovery using matrix factorization. Proc Natl Acad Sci USA 101(12): 4164–4169

    Article  Google Scholar 

  22. Cichocki A, Amari S, Siwek K, Tanaka T (2006) The ICALAB package: for image processing, version 1.2, RIKEN Brain Science Institute, Wako shi

Download references

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 610054, People’s Republic of China

    Shangming Yang & Mao Ye

Authors
  1. Shangming Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mao Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shangming Yang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Yang, S., Ye, M. Global Minima Analysis of Lee and Seung’s NMF Algorithms. Neural Process Lett 38, 29–51 (2013). https://doi.org/10.1007/s11063-012-9261-x

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-012-9261-x

Keywords

Search

Navigation