Skip to main content
Springer Nature Link
Log in

Solving non-negative matrix factorization by alternating least squares with a modified strategy

  • Published:
Data Mining and Knowledge Discovery Aims and scope Submit manuscript

Abstract

Non-negative matrix factorization (NMF) is a method to obtain a representation of data using non-negativity constraints. A popular approach is alternating non-negative least squares (ANLS). As is well known, if the sequence generated by ANLS has at least one limit point, then the limit point is a stationary point of NMF. However, no evdience has shown that the sequence generated by ANLS has at least one limit point. In order to overcome this shortcoming, we propose a modified strategy for ANLS in this paper. The modified strategy can ensure the sequence generated by ANLS has at least one limit point, and this limit point is a stationary point of NMF. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  • Ahn J-H, Kim S-K, Oh J-H, Choi S (2004) Multiple nonnegativematrix factorization of dynamic PET images. In ACCV

  • Brunet J-P, Tamayo P, Golub T R, Mesirov J P (2004) Metagenes and molecular pattern discovery using matrix factorization. Proc Nat Acad Sci USA 101: 4164–4169

    Article  Google Scholar 

  • Cho Y-C, Choi S (2005) Nonnegative features of spectro-temporal sounds for classification. Pattern Recognit Lett 26: 1327–1336

    Article  Google Scholar 

  • Chu M, Diele F, Plemmons R, Ragni S (2004) Optimality, computation, and interpretations of nonnegative matrix factorizations. SIAM J Matrix Anal 4–8030

  • Gao Y, Church G (2005) Improving molecular cancer class discovery through sparse non-negative matrix factorization. Bioinformatics 21: 3970–3975

    Article  Google Scholar 

  • Gonzalez EF, Zhang Y (2005) Accelerating the Lee-Seung algorithm for nonnegative matrix factorization, Technical report, Rise University

  • Guillamet D, Vitria J (2002) Classifying faces with nonnegative matrix factorization. In Proceedings of the Fifth Catalan Conference for Artificial Intelligence

  • Guillamet D, Schiele B, Vitria J (2002) Analyzing non-negative matrix factorization for image classification. 16th International Conference on Pattern Recognition, vol 2, pp 116–119

  • Guillamet D, Vitria J, Schiele B (2003) Introducing a weighted non-negative matrix factorization for image classification. Pattern Recognit Lett 24: 2447–2454

    Article  MATH  Google Scholar 

  • Kim H, Park H (2007) Sparse non-negative matrix factorizations via alternating non-negativity-constrained least squares for microarray data analysis. Bioinformatics 23: 1495–1502

    Article  Google Scholar 

  • Kim H, Park H (2008) Nonnegative matrix factorization based on alternating nonnegativity constrained least squares and active set method. SIAM J Matrix Anal Appl 30: 713–730

    Article  MathSciNet  MATH  Google Scholar 

  • Kim PM, Tidor B (2003) Subsystem identification through dimensionality reduction of large-scale gene expression data. Genome Res 13: 1706–1718

    Article  Google Scholar 

  • Kim D, Sra S, Dhillon IS (2007) Fast newton-type methods for the least squares nonnegative matrix approximation problem. In: Proceedings of SIAM Conference on Data Mining

  • Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401: 788–791

    Article  Google Scholar 

  • Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Proceedings of Neural Information Processing Systems, vol 13, pp 556–562

  • Lee JS, Lee DD, Choi S, Lee DS (2002) Application of nonnegative matrix factorization to dynamic positron emission tomography. In 3rd International Conference on Independent Component Analysis and Blind Signal Separation, pp 556–562

  • Li H, Adali T, Wang W, Emge D (2005) Non-negative matrix factorization with orthogonality constraints for chemical agent detection in raman spectra. Machine Learning for Signal Processing, pp 253–258

  • Lin C-J (2007) Projected gradient methods for nonnegative matrix factorization. Neural Comput 19: 2756–2779

    Article  MathSciNet  MATH  Google Scholar 

  • Lin C-J (2007) On the convergence of multiplicative update algorithms for non-negative matrix factorization. IEEE Transac Neural Netw 18: 1589–1596

    Article  Google Scholar 

  • Liu W, Zheng N (2004) Non-negative matrix factorization based methods for object recognition. Pattern Recognit Lett 25: 893–897

    Article  Google Scholar 

  • Okun O, Priisalu H (2009) Fast nonnegative matrix factorization and its application for protein fold recognition. EURASIP J Appl Signal Process 2006: 62

    Google Scholar 

  • Paatero P (1999) The multilinear enginea table-driven least squares program for solving multilinear problems, including the n-way parallel factor analysis model. J Comput Graph Stat 8: 1–35

    MathSciNet  Google Scholar 

  • Paatero P, Tapper U (1994) Positive matrix factorization: a non-negative factor model with optimal utilization of error estimates of data values. Environmetrics 5: 111–126

    Article  Google Scholar 

  • Pascual-Montano A, Carazo JM, Kochi K, Lehmann D, Pascual-Marqui RD (2006) Nonsmooth nonnegative matrix factorization (nsNMF). IEEE Transac Pattern Anal Mach Intell 28: 403–415

    Article  Google Scholar 

  • Paul Pauca V, Shahnaz F, Berry Michael W, Plemmons Robert J (2004) Text mining using non-negative matrix factorization. In: Proceedings of the Fourth SIAM International Conference on Data Mining 2004

  • Rao N, Shepherd SJ (2004) Extracting characteristic patterns from genomewide expression data by non-negative matrix factorization. In Computational Systems Bioinformatics Conference

  • Richardson WH (1972) Bayesian-based iterative method of image restoration. J Opt Soc Am 62: 55–59

    Article  Google Scholar 

  • Spratling MW (2006) Learning image components for object recognition. J Mach Learn Res 7: 793–815

    MathSciNet  MATH  Google Scholar 

  • Wang Y, Jia Y, Hu C, Hu C (2005) Non-negative matrix factorization framework for face recognition. Int J Pattern Recognit Artif Intell 19: 495–511

    Article  Google Scholar 

  • Zdunek R, Cichocki A (2006) Non-negative matrix factorization with quasi-newton optimization. International Conference on Artificial Intelligence and Soft Computing (ICAISC 2006), pp 870–879

  • Zdunek R, Cichocki A (2008) Fast nonnegativematrix factorization algorithms using projected gradient approaches for large-scale problems. Hindawi Publishing Corporation Computational Intelligence and Neuroscience

Download references

Author information

Authors and Affiliations

  1. Department of Mathematics, Xidian University, Xi’an, 710071, People’s Republic of China

    Hongwei Liu, Xiangli Li & Xiuyun Zheng

  2. College of Mathematics and Computing Science, Guilin University of Electronic Technology, Guilin, 541004, People’s Republic of China

    Xiangli Li

Authors
  1. Hongwei Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiangli Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xiuyun Zheng
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiangli Li.

Additional information

Responsible editor: Chih-Jen Lin.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Liu, H., Li, X. & Zheng, X. Solving non-negative matrix factorization by alternating least squares with a modified strategy. Data Min Knowl Disc 26, 435–451 (2013). https://doi.org/10.1007/s10618-012-0265-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10618-012-0265-y

Keywords