Skip to main content
SpringerLink
Log in

Semi-supervised Nonnegative Matrix Factorization with Commonness Extraction

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Standard nonnegative matrix factorization extracts nonnegative bases for nonnegative representation, which, however, considers only features, not commonness. In addition, the standard NMF is an unsupervised learning method that cannot fully utilize label information if it exists. In this paper, we present a semi-supervised commonness NMF technique that incorporates samples’ commonness and label information into the optimization model. Naturally, the commonness vector should be constrained by nonnegativity and will degenerate to zero if no commonness exists. We develop a multiplicative update rule to solve the model, which has properties comparable to those of the standard NMF with automatic satisfaction of the nonnegativity constraints, monotonicity without the need for any adjustable learning rate and a low computational overhead. Through experiments on the standard databases, we analyze the behavior of the proposed method, which exhibits a performance that is favorably superior with respect to commonness extraction and clustering accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. Lee et al. introduced two weighted matrices to handle missing entries and labels of the data. The former may be used to all NMF methods, which, however, is not considered in the scope of this paper. And the latter is incorporated to the matrix L for convenience.

References

  1. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401:788–791

    Article  Google Scholar 

  2. Lee DD, Seung HS (2000) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp. 556–562

  3. Hoyer P (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469

    MathSciNet  MATH  Google Scholar 

  4. Dhillon IS, Sra S (2005) Generalized nonnegative matrix approximations with bregman divergences. In: Proceedings of the neural information processing systems, pp. 283–290

  5. Cichocki A, Zdunek R, Amari S (2006) Csiszar’s divergences for non-negative matrix factorization: family of new algorithms. Lect Notes Comput Sci 3889:32–39

    Article  MATH  Google Scholar 

  6. Cichockia A, Lee H, Kimb YD, Choi SJ (2008) Non-negative matrix factorization with \(\alpha \)-divergence. Pattern Recogn Lett 29:1433–1440

    Article  Google Scholar 

  7. Fevotte C, Bertin N, Durrieu J (2009) Nonnegative matrix factorization with the itakura-saito divergence: with application to music analysis. Neural Computat 2:793–830

    Article  MATH  Google Scholar 

  8. Cho Y, Saul LK (2011) Nonnegative matrix factorization for semi-supervised dimensionality reduction. arXiv:1112.3714

  9. Mohammadiha N, Smaragdis P, Leijon A (2007) Supervised and unsupervised speech enhancement using nonnegative matrix factorization. IEEE Trans Audio Speech 21:2140–2151

    Article  Google Scholar 

  10. Wang D, Gao X, Wang X (2016) Semi-supervised nonnegative matrix factorization via constraint propagation. IEEE Trans Cyber 46:233–244

    Article  Google Scholar 

  11. Lee H, Yoo J, Choi S (2010) Semi-supervised nonnegative matrix factorization. IEEE Signal Process Lett 17:4–7

    Article  Google Scholar 

  12. Cichocki A, Amari S, Zdunek R, Kompass P, Hori G, He Z (2006) Extended smart algorithms for non-negative matrix factorization. Lect Notes Artif Intell 4029:548–562

    Google Scholar 

  13. Zdunek R, Cichocki A (2007) Gibbs regularized nonnegative matrix factorization for blind separation of locally smooth signals. In: Proceedings of the 15th IEEE international workshop on nonlinear dynamics of electronic systems (NDES07), pp. 317–320

  14. Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33:1548–1560

    Article  Google Scholar 

  15. Liu H, Wu Z, Li X, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34:233–244

    Google Scholar 

  16. Wang C, Yan S, Zhang L, Zhang H (2009) Non-negative semi-supervised learning. In: Proceedings of the twelfth international conference on artificial intelligence and statistics, pp 575–582

  17. Ma H, Jia M, Shi YK, Hao Z, Liu Y, Jin R, Yang L (2014) Semi-supervised multi-label learning by constrained non-negative matrix factorization. In: proceedings of the national conference on artificial intelligence 18th innovative applications of artificial intelligence conference, Boston, pp. 511–516

  18. Kannan R, Ballard G, Park H (2015) A high-performance parallel algorithm for nonnegative matrix factorization. arXiv:1509.09313

  19. Gillis N, Kumar A (2015) Exact and heuristic algorithms for semi-nonnegative matrix factorization. SIAM J Matrix Anal Appl 36:1404–1424

    Article  MathSciNet  MATH  Google Scholar 

  20. Gillis N, Vavasis SA (2015) Semidefinite programming based preconditioning for more robust near-separable nonnegative matrix factorization. SIAM J Opt 25:677–698

    Article  MathSciNet  MATH  Google Scholar 

  21. Vandaele A, Gillis N, Lei Q, Zhong K, Dhillon I (2016) Efficient and non-convex coordinate descent for symmetric nonnegative matrix factorization. IEEE Trans Signal Process 64:5571–5584

    Article  MathSciNet  Google Scholar 

  22. Bhattacharya C, Goyal N, Kannan R, Pani J (2016) Non-negative matrix factorization under heavy noise. In: Proceedings of the 33rd international conference on machine learning, New York, pp. 1426–1434

  23. Samaria F, Harter A (1994) Parameterisation of a stochastic model for human face identification. In: Proceedings of 2nd IEEE workshop applications of computer vision, pp. 5–7

  24. Bellhumer PN, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 17:711–720

    Article  Google Scholar 

  25. Nene SA, Nayar SK, Murase H (1996) Columbia object image library (COIL-100), Technical reports, CUCS-006-96, February, 1996

  26. Lichman M (2013) UCI machine learning repository. University of California, School of Information and Computer Science, Irvine. http://archive.ics.uci.edu/ml

  27. Odewahn S, Stockwell E, Pennington R, Humphreys R, Zunach W (1992) Automated star/galaxy discrimination with neural networks. Astron J 103:318–331

    Article  Google Scholar 

  28. Lovasz L, Plummer M (1986) Matching theory, Amsterdam. Elsevier, The Netherlands

    MATH  Google Scholar 

  29. Stone M (1974) Cross-validatory choice and assessment of statistical predictions. J R Stat Soc 36:111–147

    MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

This work was supported by the National Natural Science Foundations of China (61302013, 61372014, 61374015), Science and Technology Plan of Liaoning Province of China (2014305001) and Fundamental Research Funds for the Central Universities of China (N141008001).

Author information

Authors and Affiliations

  1. Sino-Dutch Biomedical and Information Engineering School, Northeastern University, Shenyang, 110004, China

    Yueyang Teng, Shouliang Qi, Yin Dai, Lisheng Xu, Wei Qian & Yan Kang

Authors
  1. Yueyang Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Shouliang Qi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yin Dai
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lisheng Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Wei Qian
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Yan Kang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yueyang Teng.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Teng, Y., Qi, S., Dai, Y. et al. Semi-supervised Nonnegative Matrix Factorization with Commonness Extraction. Neural Process Lett 45, 1063–1076 (2017). https://doi.org/10.1007/s11063-016-9565-3

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-016-9565-3

Keywords

Search

Navigation