Skip to main content
SpringerLink
Log in

Graph based semi-supervised learning via label fitting

  • Original Article
  • Published:
International Journal of Machine Learning and Cybernetics Aims and scope Submit manuscript

Abstract

The global smoothness and the local label fitting are two key issues for estimating the function on the graph in graph based semi-supervised learning (GSSL). The unsupervised normalized cut method can provide a more reasonable criterion for learning the global smoothness of the data than classic GSSL methods. However, the semi-supervised norm of the normalized cut, which is a NP-hard problem, has not been studied well. In this paper, a new GSSL framework is proposed by extending normalized cut to its semi-supervised norm. The NP-hard semi-supervised normalized cut problem is innovatively solved by effective algorithms. In addition, we can design more reasonable local label fitting terms than conventional GSSL methods. Other graph cut methods are also investigated to extend the proposed semi-supervised learning algorithms. Furthermore, we incorporate the nonnegative matrix factorization with the proposed learning algorithms to solve the out-of-sample problem in semi-supervised learning. Solutions obtained by the proposed algorithms are sparse, nonnegative and congruent with unit matrix. Experiment results on several real benchmark datasets indicate that the proposed algorithms achieve good results compared with state-of-art methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. http://cvc.yale.edu/projects/yalefaces/yalefaces.html.

  2. http://www-i6.informatik.rwth-aachen.de/~keysers/usps.html.

  3. http://www.zjucadcg.cn/dengcai/GNMF/.

  4. http://vision.ucsd.edu/~leekc/ExtYaleDatabase/ExtYaleB.html.

References

  1. Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res (JAIR) 23:331–366

    MATH  Google Scholar 

  2. Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MA

    Book  Google Scholar 

  3. Zhu X, Goldberg AB (2009) Introduction to semi-supervised learning. Synth Lect Artif Intell Mach Learn 3(1):1–130

    Article  MATH  Google Scholar 

  4. Nie F, Xu D, Tsang IWH, Zhang C (2010) Flexible manifold embedding: a framework for semi-supervised and unsupervised dimension reduction. Image Process IEEE Trans 19(7):1921–1932

    Article  MathSciNet  Google Scholar 

  5. Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76

    Article  Google Scholar 

  6. Cai D, He X, Han J (2007) Semi-supervised discriminant analysis. In: Computer Vision, 2007. ICCV 2007. IEEE 11th international conference on IEEE, pp 1–7

  7. Zhou D, Bousquet O, Lal TN, Weston J, Schölkopf B (2004) Learning with local and global consistency. Adv Neural Inf Process Syst 16(16):321–328

    Google Scholar 

  8. Wang J, Jebara T, Chang SF (2013) Semi-supervised learning using greedy max-cut. J Mach Learn Res 14(1):771–800

    MathSciNet  MATH  Google Scholar 

  9. Zhu X, Ghahramani Z, Lafferty J (2003) Semi-supervised learning using gaussian fields and harmonic functions. In: ICML, vol 3, pp 912–919

  10. Tang J, Hua XS, Qi GJ, Wang M, Mei T, Wu X (2007) Structure-sensitive manifold ranking for video concept detection. In: Proceedings of the 15th international conference on Multimedia, ACM, pp 852–861

  11. Tang J, Hua XS, Qi GJ, Song Y, Wu X (2008) Video annotation based on kernel linear neighborhood propagation. Multimed IEEE Trans 10(4):620–628

    Article  Google Scholar 

  12. Wang M, Mei T, Yuan X, Song Y, Dai LR (2007) Video annotation by graph-based learning with neighborhood similarity. In: Proceedings of the 15th international conference on multimedia, ACM, pp 325–328

  13. Zhao M, Chow TW, Zhang Z, Li B (2015) Automatic image annotation via compact graph based semi-supervised learning. Knowl-Based Syst 76:148–165

    Article  Google Scholar 

  14. Huang L, Wang Y, Liu X, Lang B (2013) Efficient semi-supervised annotation with proxy-based local consistency propagation. In: Multimedia and Expo (ICME), 2013 IEEE international conference on, IEEE, pp 1–6

  15. Liu S, Yan S, Zhang T, Xu C, Liu J, Lu H (2012) Weakly supervised graph propagation towards collective image parsing. Multimed IEEE Trans 14(2):361–373

    Article  Google Scholar 

  16. Zhao M, Chan RH, Chow TW, Tang P (2014) Compact graph based semi-supervised learning for medical diagnosis in Alzheimer’s disease. Signal Process Lett IEEE 21(10):1192–1196

    Article  Google Scholar 

  17. Wang F, Zhang C (2008) Label propagation through linear neighborhoods. Knowl Data Eng IEEE Trans 20(1):55–67

    Article  Google Scholar 

  18. Zhuang L, Gao H, Lin Z, Ma Y, Zhang X, Yu N (2012) Non-negative low rank and sparse graph for semi-supervised learning. In: Computer vision and pattern recognition (CVPR), 2012 IEEE conference on, IEEE, pp 2328–2335

  19. Ni B, Yan S, Kassim A (2012) Learning a propagable graph for semisupervised learning: classification and regression. Knowl Data Eng IEEE Trans 24(1):114–126

    Article  Google Scholar 

  20. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  MATH  Google Scholar 

  21. Shi J, Malik J (2000) Normalized cuts and image segmentation. Pattern Anal Mach Intell IEEE Trans 22(8):888–905

    Article  Google Scholar 

  22. Hagen L, Kahng AB (1992) New spectral methods for ratio cut partitioning and clustering. Comput Aid Des Integr Circuit Syst IEEE Trans 11(9):1074–1085

    Article  Google Scholar 

  23. Sarkar S, Soundararajan P (2000) Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. Pattern Anal Mach Intell IEEE Trans 22(5):504–525

    Article  Google Scholar 

  24. Wu Z, Leahy R (1993) An optimal graph theoretic approach to data clustering: theory and its application to image segmentation. Pattern Anal Mach Intell IEEE Trans 15(11):1101–1113

    Article  Google Scholar 

  25. Ding CH, He X, Zha H, Gu M, Simon HD (2001) A min–max cut algorithm for graph partitioning and data clustering. In: Data mining, 2001. ICDM 2001, proceedings IEEE international conference on, IEEE, pp 107–114

  26. Xu L, Li W, Schuurmans D (2009) Fast normalized cut with linear constraints. In: Computer vision and pattern recognition, IEEE conference on, IEEE, pp 2866–2873

  27. Hu H, Feng J, Yu C, Zhou J (2013) Multi-class constrained normalized cut with hard, soft, unary and pairwise priors and its applications to object segmentation. Image Process IEEE Trans 22(11):4328–4340

    Article  MathSciNet  Google Scholar 

  28. Yang YT, Fishbain B, Hochbaum DS, Norman EB, Swanberg E (2013) The supervised normalized cut method for detecting, classifying, and identifying special nuclear materials. INFORMS J Comput 26(1):45–58

    Article  Google Scholar 

  29. Kulis B, Basu S, Dhillon I, Mooney R (2009) Semi-supervised graph clustering: a kernel approach. Mach Learn 74(1):1–22

    Article  Google Scholar 

  30. Ren W, Li G, Tu D, Jia L (2014) Nonnegative matrix factorization with regularizations. Emerg Select Topn Circuit Syst IEEE J 4(1):153–164

    Article  Google Scholar 

  31. Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416

    Article  MathSciNet  Google Scholar 

  32. Ding CH, He X, Simon HD (2005) On the equivalence of nonnegative matrix factorization and spectral clustering. In: SDM, vol 5, pp 606–610

  33. Ren WY, Li GH, Tu D (2015) Graph clustering by congruency approximation. IET Comput Vis. doi:10.1049/iet-cvi.2014.0131

    Google Scholar 

  34. Kulis B (2012) Metric learning: a survey. Found Trend Mach Learn 5(4):287–364

    Article  MATH  Google Scholar 

  35. James W, Stein C (1961) Estimation with quadratic loss. In: Proceedings of the fourth Berkeley symposium on mathematical statistics and probability, vol 1 pp 361–379

  36. Kjeldsen TH (2000) A contextualized historical analysis of the Kuhn–Tucker Theorem in nonlinear programming: the impact of World War II. Hist Math 27(4):331–361

    Article  MathSciNet  MATH  Google Scholar 

  37. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the 10th international workshop on artificial intelligence and statistics, vol 1, pp 57–64

  38. Niyogi X (2004) Locality preserving projections. In: Neural information processing systems, vol 16, p 153. MIT

  39. Liu G, Lin Z, Yan S, Sun J, Yu Y, Ma Y (2013) Robust recovery of subspace structures by low-rank representation. Pattern Anal Mach Intell IEEE Trans 35(1):171–184

    Article  Google Scholar 

  40. Hoyer PO (2004) Non-negative matrix factorization with sparseness constraints. J Mach Learn Res 5:1457–1469

    MathSciNet  MATH  Google Scholar 

  41. Fujiwara Y, Irie G (2014) Efficient label propagation. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 784–792

  42. Lee DD, Seung HS (1999) Learning the parts of objects by non-negative matrix factorization. Nature 401(6755):788–791

    Article  Google Scholar 

  43. Lee DD, Seung HS (2001) Algorithms for non-negative matrix factorization. In: Advances in neural information processing systems, pp 556–562

  44. Cai D, He X, Wu X, Han J (2008) Non-negative matrix factorization on manifold. In: Data mining, 2008. ICDM’08, 8th IEEE international conference on (pp 63–72), IEEE

  45. Cai D, He X, Han J, Huang TS (2011) Graph regularized nonnegative matrix factorization for data representation. Pattern Anal Mach Intell IEEE Trans 33(8):1548–1560

    Article  Google Scholar 

  46. Zadeh LA (1968) Probability measures of fuzzy events. J Math Anal Appl 23(2):421–427

    Article  MathSciNet  MATH  Google Scholar 

  47. Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23(5):1638–1654

    Article  Google Scholar 

  48. Wang XZ, Aamir Raza Ashfaq R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29(3):1185–1196

    Article  MathSciNet  Google Scholar 

  49. Wang XZ, Dong LC, Yan JH (2012) Maximum ambiguity-based sample selection in fuzzy decision tree induction. Knowl Data Eng IEEE Trans 24(8):1491–1505

    Article  Google Scholar 

  50. Shannon CE (1948) A mathematical theory of communication. Bell Syst Tech J 27:379–423, 623–656

    Article  MathSciNet  MATH  Google Scholar 

  51. Hartley RVL (1949) Transmission of information. Bell Syst Tech J 7:535–563

    Article  Google Scholar 

Download references

Acknowledgments

This paper is supported by College of Information System and Management, National University of Defense Technology and subsidized by National Natural Science Foundation of China Grant No. 61170158.

Author information

Authors and Affiliations

  1. College of Information System and Management, National University of Defense Technology, Changsha, 410072, People’s Republic of China

    Weiya Ren & Guohui Li

Authors
  1. Weiya Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Guohui Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Weiya Ren.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ren, W., Li, G. Graph based semi-supervised learning via label fitting. Int. J. Mach. Learn. & Cyber. 8, 877–889 (2017). https://doi.org/10.1007/s13042-015-0458-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s13042-015-0458-y

Keywords

Search

Navigation