Skip to main content
Log in

Semi-supervised classification of multiple kernels embedding manifold information

  • Published:
Cluster Computing Aims and scope Submit manuscript

Abstract

For semi-supervised learning, we propose the Laplacian embedded multiple kernel regression model. As we incorporate the multiple kernel occasion into manifold regularization framework, the models we proposed are flexible in many kinds of datasets and have a solid theoretical foundation. The proposed model can solve the two problems, which are the computation cost of manifold regularization framework and the difficulty in dealing with multi-source or multi-attribute datasets. Though manifold regularization is a convex optimization formulation, it often leads to dense matrix inversion with computation cost. Laplacian embedded method we adopted can solve the problem, however it lacks the proper ability to process complex datasets. Therefore, we further use multiple kernel learning as a part of the proposed model to strengthen its ability. Experiments on several datasets compared with the state-of-the-art methods show the effectiveness and efficiency of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Zhu, X.: Semi-supervised learning literature survey. Comput. Sci. 37(1), 63–77 (2005)

    Google Scholar 

  2. Zhu, S., Sun, X., Jin, D.: Multi-view semi-supervised learning for image classification. Neurocomputing 208, 136–142 (2016)

    Article  Google Scholar 

  3. Amini, M.R., Usunier, N.: Semi-supervised learning. Intell. Syst. Ref. Libr. 49(2), 215–239 (2015)

    MATH  Google Scholar 

  4. Mehrkanoon, S., Alzate, C., Mall, R., et al.: Multiclass semisupervised learning based upon kernel spectral clustering. IEEE Trans. Neural Netw. Learn. Syst. 26(4), 720–733 (2015)

    Article  MathSciNet  Google Scholar 

  5. Lu, Z., Wang, L.: Noise-robust semi-supervised learning via fast sparse coding. Pattern Recognit. 48(2), 605–612 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Chapelle, O., Weston, J., Schölkopf, B.: Cluster kernels for semi supervised learning. Adv. Neural Inf. Process. Syst. 585–592 (2002)

  7. Chapelle, O., Schölkopf, B., Zien, A.: Semi-supervised learning. MIT Press, Cambridge (2006)

    Book  Google Scholar 

  8. Joachims, T.: Transductive inference for text classification using support vector machines. Proc. Int. Conf. Mach. Learn. 99, 200–209 (1999)

    Google Scholar 

  9. Bennett, K., Demiriz, A.: Semi-supervised support vector machines. Adv. Neural Inf. Process. Syst. 368–374 (1999)

  10. Chapelle, O., Zien, A.: Semi-supervised classification by low density separation. Proc. Int. Workshop Artif. Intell. Stat. 57–64 (2005)

  11. Yanai, K.: Tools on support vector machines: SVMLight, LIBSVM, SHOGUN. J. Inst. Image Inf. Telev. Eng. 63, 1778–1781 (2009)

    Google Scholar 

  12. Sindhwani, V., Keerthi, S.S., Chapelle, O.: Deterministic annealing for semi-supervised kernel machines. In: Proceedings of the International Conference on Machine Learning. ACM, pp. 841–848 (2006)

  13. Collobert, R., Sinz, F., Weston, J., et al.: Large scale transductive SVMs. J. Mach. Learn. Res. 7, 1687–1712 (2006)

    MATH  MathSciNet  Google Scholar 

  14. Zhao, H.: Combining labeled and unlabeled data with graph embedding. Neurocomputing 69(16), 2385–2389 (2006)

    Article  Google Scholar 

  15. Zhu, X., Ghahramani, Z., Lafferty, J.: Semi-supervised learning using gaussian fields and harmonic functions. Proc. Int. Conf. Mach. Learn. 3, 912–919 (2003)

    Google Scholar 

  16. Zhou, D., Bousquet, O., Lal, T.N., et al.: Learning with local and global consistency. Adv. Neural Inf. Process. Syst. 16, 321–328 (2003)

    Google Scholar 

  17. Belkin, M., Niyogi, P., Sindhwani, V.: Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J. Mach. Learn. Res. 7, 2399–2434 (2006)

    MATH  MathSciNet  Google Scholar 

  18. Chen, L., Tsang, I.W., Xu, D.: Laplacian embedded regression for scalable manifold regularization. IEEE Trans. Neural Netw. Learn. Syst. 23(6), 902–915 (2012)

    Article  Google Scholar 

  19. Zhang, K., Wang, Q., Lan, L., et al.: Sparse semi-supervised learning on low-rank kernel. Neurocomputing 129(4), 265–272 (2014)

    Article  Google Scholar 

  20. Zhang, Z., Zhang, Y., Li, F.Z., et al.: Discriminative sparse flexible manifold embedding with novel graph for robust visual representation and label propagation. Pattern Recognit. 61(1), 492–510 (2017)

    Article  Google Scholar 

  21. Yu, J., Rui, Y., Chen, B.: Exploiting click constraints and multi-view features for image re-ranking. IEEE Trans. Multimed. 16(1), 159–1682 (2013)

    Article  Google Scholar 

  22. Yu, J., Tao, D., Rui, Y., et al.: Pairwise constraints based multiview features fusion for scene classification. Pattern Recognit. 46(2), 483–496 (2013)

  23. Zhou, D., Huang, J., Schölkopf, B.: Learning with hypergraphs: clustering, classification, and embedding. Adv. Neural Inf. Process. Syst. 19, 1601–1608 (2007)

    Google Scholar 

  24. Paulsen, V.I., Raghupathi, M.: An Introduction to the Theory of Reproducing Kernel Hilbert Spaces. Cambridge University Press, Cambridge (2016)

    Book  MATH  Google Scholar 

  25. Gönen, M., Alpaydın, E.: Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2268 (2011)

    MATH  MathSciNet  Google Scholar 

  26. Lanckriet, G.R.G., Cristianini, N., Bartlett, P., et al.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)

    MATH  MathSciNet  Google Scholar 

  27. Zare, T., Sadeghi, M.T.: A novel multiple kernel-based dictionary learning for distributive and collective sparse representation based classifiers. Neurocomputing 234, 164–173 (2017)

    Article  Google Scholar 

  28. Rakotomamonjy, A., Bach, F.R., Canu, S., et al.: SimpleMKL. J. Mach. Learn. Res. 9, 2491–2521 (2008)

    MATH  MathSciNet  Google Scholar 

  29. Lu, M., Yang, L., Wang, J., et al.: Applications of pointgroup density cartography based on kernel density estimation. Eng. Surv. Mapp. (2017)

  30. Duan, L., Tsang, I.W., Xu, D.: Domain transfer multiple kernel learning. IEEE Trans. Pattern Anal. Mach. Intell. 34(3), 465–479 (2011)

    Article  Google Scholar 

  31. Kumar A, Niculescu-Mizil A, Kavukcuoglu K, et al. A binary classification framework for two-stage multiple kernel learning[J]. arXiv preprint arXiv:1206.6428, 2012

  32. Suzuki, T., Tomioka, R.: SpicyMKL: a fast algorithm for multiple kernel learning with thousands of kernels. Mach. Learn. 85(1–2), 77–108 (2011)

    Article  MATH  MathSciNet  Google Scholar 

  33. Cortes, C., Mohri, M., Rostamizadeh, A.: Learning non-linear combinations of kernels. Adv. Neural Inf. Process. Syst. 396–404 (2009)

  34. Kloft, M., Brefeld, U., Sonnenburg, S., et al.: Non-sparse regularization and efficient training with multiple kernels. (2010). arXiv:1003.0079

  35. Jain, A., Vishwanathan, S.V.N., Varma, M.: SPF-GMKL generalized multiple kernel learning with a million kernels. In: Proceedings of the International Conference on Knowledge Discovery and Data Mining. ACM, pp. 750-758 (2012)

  36. Cortes, C., Mohri, M., Rostamizadeh, A.: Generalization bounds for learning kernels. Proc. Int. Conf. Mach. Learn. pp. 247–254 (2010)

  37. Cortes, C., Kloft, M., Mohri, M.: Learning kernels using local Rademacher complexity. Adv. Neural Inf. Process. Syst. 2760–2768 (2013)

  38. Schölkopf, B., Herbrich, R., Smola, A.J.: A generalized representer theorem, pp. 416–426. In: Computational learning theory. Springer, Berlin (2001)

  39. Yufeng, L., James, T.K., Zhihua, Z.: Semi-supervised learning using label mean. In: Proceedings of the International Conference on Machine Learning. ACM, pp. 633–640 (2009)

  40. Bache, K., Lichman, M.: UCI Machine Learning Repository. University of California, School of Information and Computer Science, Irvine (2013)

    Google Scholar 

  41. Griffin, G., Holub, A., Perona, P.: Caltech-256 Object Category Dataset. California Institute of Technology, Pasadena (2007)

    Google Scholar 

  42. Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2, 27 (2011)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported by Fundamental Research Funds for the Central Universities (No. FRF-TP-16-082A1).

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Tao Yang.

Appendix

Appendix

Full name

Abbreviation

Full name

Abbreviation

Semi-supervised learning

SSL

Support vector machine

SVM

Semi-supervised support vector machine

S\(^{3}\)VM

Mean Semi-supervised support vector machine

MeanS\(^{3}\)VM

Gaussian random field

GRF

K-nearest neighbor

KNN

Low-density separation

LDS

Deterministic annealing

DA

Laplacian support vector machine

LapSVM

Manifold regularization

MR

Reproducing kernel Hilbert space

RKHS

Laplacian embedded support vector regression

LapESVR

Laplacian embedded multiple kernel regression

LapEMKR

Multiple kernel learning

MKL

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, T., Fu, D. & Li, X. Semi-supervised classification of multiple kernels embedding manifold information. Cluster Comput 20, 3417–3426 (2017). https://doi.org/10.1007/s10586-017-1123-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10586-017-1123-x

Keywords

Navigation