Skip to main content
SpringerLink
Log in

Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Non-negative matrix factorization (NMF) plays an important role in multivariate data analysis, and has been widely applied in information retrieval, computer vision, and pattern recognition. NMF is an effective method to capture the underlying structure of the data in the parts-based low dimensional representation space. However, NMF is actually an unsupervised method without making use of supervisory information of data. In recent years, semi-supervised learning has received a lot of attentions, because partial label information can significantly improve learning quality of the algorithms. In this paper, we propose a novel semi-supervised non-negative matrix factorization (SEMINMF) algorithm, which not only utilizes the local structure of the data characterized by the graph Laplacian, but also incorporates the label information as the fitting constraints to learn. Hence, it can learn from labeled and unlabeled data. By this means our SEMINMF can obtain a more discriminative powerful representation space. Experimental results show the effectiveness of our proposed novel method in comparison to the state-of-the-art algorithms on several real world applications.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Notes

  1. http://en.wikipedia.org/wiki/Caltech_101

References

  1. Bach F, Jordan M (2003) Learning spectral clustering. Computer Science Division, University of California

  2. Basu S, Bilenko M, Mooney R et al (2004) A probabilistic framework for semi-supervised clustering. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, pp 22–25

  3. Cai D, He X, Han J, Huang T (2011) Graph regularized nonnegative matrix factorization for data representation. IEEE Trans Pattern Anal Mach Intell 33(8):1548–1560

    Article  Google Scholar 

  4. Chapelle O, Schölkopf B, Zien A et al (2006) Semi-supervised learning, vol 2. MIT Press, Cambridge, MA

    Book  Google Scholar 

  5. Chen Y, Rege M, Dong M, Hua J (2008) Non-negative matrix factorization for semi-supervised data clustering. Knowl Inf Syst 17(3):355–379

    Article  Google Scholar 

  6. Chung F (1997) Spectral graph theory, no 92. American Mathematical Society

  7. Cormen T (2001) Introduction to algorithms. The MIT Press

  8. Das Gupta M, Xiao J (2011) Non-negative matrix factorization as a feature selection tool for maximum margin classifiers. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR). IEEE, pp 2841–2848

  9. De la Torre F, Kanade, T (2006) Discriminative cluster analysis. In: Proceedings of the 23rd international conference on machine learning. ACM, pp 241–248

  10. Dempster A, Laird N, Rubin D (1977) Maximum likelihood from incomplete data via the em algorithm. J Roy Stat Soc Ser B (Methodological) 39:1–38

    MATH  MathSciNet  Google Scholar 

  11. Ding C, Li T (2007) Adaptive dimension reduction using discriminant analysis and k-means clustering. In: ACM international conference proceeding series, vol 227, pp 521–528

  12. Ding C, Li T, Jordan M (2008) Nonnegative matrix factorization for combinatorial optimization: spectral clustering, graph matching and clique finding. In: 8th IEEE International Conference on Data Mining, 2008. ICDM’08. IEEE, pp 183–192

  13. Guillamet D, Vitria J (2002) Classifying faces with nonnegative matrix factorization. In: Proc. 5th Catalan conference for artificial intelligence

  14. Hoyer P (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469

    MATH  MathSciNet  Google Scholar 

  15. Kim J, Park, H (2008) Sparse nonnegative matrix factorization for clustering.

  16. Lee D, Seung H (2001) Algorithms for non-negative matrix factorization. Adv Neural Inf Process Syst 13:556–562

    Google Scholar 

  17. Lee H, Yoo J, Choi S (2010) Semi-supervised nonnegative matrix factorization. IEEE Signal Process Lett 17(1):4–7

    Google Scholar 

  18. Li T, Ding C (2006) The relationships among various nonnegative matrix factorization methods for clustering. In: 6th International conference on data mining, 2006. ICDM’06. IEEE, pp 362–371

  19. Lin TC, Huang HC, Liao BY, Pan JS (2007) An optimized approach on applying genetic algorithm to adaptive cluster validity index. Int J Comput Sci Eng Syst 1(4):253–257

    MATH  Google Scholar 

  20. Liu H, Wu Z (2010) Non-negative matrix factorization with constraints. In: 24th AAAI conference on artificial intelligence

  21. Liu H, Wu Z, Li X, Cai D, Huang TS (2012) Constrained nonnegative matrix factorization for image representation. IEEE Trans Pattern Anal Mach Intell 34(7):1299–1311

    Article  Google Scholar 

  22. Lovász L, Plummer M (1986) Matching theory, no 121. Elsevier Science Ltd

  23. Ma Z, Yang Y, Nie F, Uijlings J, Sebe N (2011) Exploiting the entire feature space with sparsity for automatic image annotation. In: Proceedings of the 19th ACM international conference on multimedia. ACM, pp 283–292

  24. Perona P, Zelnik-Manor L (2004) Self-tuning spectral clustering. Adv Neural Inf Process Syst 17:1601–1608

    Google Scholar 

  25. Philbin J, Chum O, Isard M, Sivic J, Zisserman A (1986) Object retrieval with large vocabularies and fast spatial matching. In: IEEE conference on computer vision and pattern recognition, 2007. CVPR’07. IEEE, pp 1–8

  26. Saul, L., Pereira, F (1997) Aggregate and mixed-order markov models for statistical language processing. In: Proceedings of the 2nd conference on empirical methods in natural language processing. Association for Computational Linguistics, Somerset, New Jersey, pp 81–89

  27. Shashua A, Hazan T (2005) Non-negative tensor factorization with applications to statistics and computer vision. In: Proceedings of the 22nd international conference on machine learning. ACM, pp 792–799

  28. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  29. Xu W, Gong Y (2004) Document clustering by concept factorization. In: Proceedings of the 27th annual international ACM SIGIR conference on research and development in information retrieval. ACM, pp 202–209

  30. Xu W, Liu X, Gong Y (2003) Document clustering based on non-negative matrix factorization. In: Proceedings of the 26th annual international ACM SIGIR conference on research and development in informaion retrieval. ACM, pp 267–273

  31. Yang Y, Xu D, Nie F, Yan S, Zhuang Y (2010) Image clustering using local discriminant models and global integration. IEEE Trans Image Process 19(10):2761–2773

    Article  MathSciNet  Google Scholar 

  32. Yang Y, Shen H, Nie F, Ji R, Zhou X (2011) Nonnegative spectral clustering with discriminative regularization. In: 25th AAAI conference on artificial intelligence. AAAI Press, pp 555–560

  33. Ye J, Zhao Z, Wu M (2007) Discriminative k-means for clustering. Adv Neural Inf Process Syst 20:1649–1656

    Google Scholar 

  34. Yu S, Shi J (2003) Multiclass spectral clustering. In: Proceedings of 9th IEEE international conference on computer vision, 2003. IEEE, pp 313–319

  35. Zhang Y, Yeung D (2008) Semi-supervised discriminant analysis using robust path-based similarity. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–8

  36. Zhang Z, Wang J, Zha H (2005) Adaptive manifold learning. IEEE Trans Pattern Anal Mach Intell 34(2):253–265

    Article  Google Scholar 

  37. Zhang Z, Zha H, Zhang M (2008) Spectral methods for semi-supervised manifold learning. In: IEEE conference on computer vision and pattern recognition, 2008. CVPR 2008. IEEE, pp 1–6

  38. Zhou D, Bousquet O, Lal T, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16:321–328

    Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Basic Research Program of China (973 program) under Grant 2009CB320901, NSFC (no. 61272247), the National High Technology Research and Development Program of China (863 program) under Grant 2008AA02Z310, the National Natural Science Foundation of China under Grant 60873133, and the Innovation Ability Special Fund of Shanghai Jiao Tong University under Grant Z030026.

Author information

Authors and Affiliations

  1. MOE-Microsoft Laboratory for Intelligent Computing and Intelligent Systems, Department of Computer Science and Engineering, Shanghai Jiao Tong University, Shanghai, 200240, People’s Republic of China

    Yangcheng He, Hongtao Lu & Saining Xie

Authors
  1. Yangcheng He
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hongtao Lu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Saining Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yangcheng He.

Rights and permissions

Reprints and permissions

About this article

Cite this article

He, Y., Lu, H. & Xie, S. Semi-supervised non-negative matrix factorization for image clustering with graph Laplacian. Multimed Tools Appl 72, 1441–1463 (2014). https://doi.org/10.1007/s11042-013-1465-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-013-1465-1

Keywords

Search

Navigation