Skip to main content
SpringerLink
Log in

Kernel Learning with Hilbert-Schmidt Independence Criterion

  • Conference paper
  • First Online:
Pattern Recognition (CCPR 2016)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 662))

Included in the following conference series:

Abstract

Measures of statistical independence between random variables have been successfully applied in many learning tasks, such as independent component analysis, feature selection and clustering. The success is based on the fact that many existing learning tasks can be cast into problems of dependence maximization (or minimization). Motivated by this, we introduce a unifying view of kernel learning with the Hilbert-Schmidt independence criterion (HSIC) which is a kernel method for measuring the statistical dependence between random variables. The key idea is that good kernels should maximize the statistical dependence, measured by the HSIC, between the kernels and the class labels. As a special case of kernel learning, we also propose an effective Gaussian kernel optimization method for classification by maximizing the HSIC, where the spherical kernel is considered. The proposed approach is demonstrated with several popular UCI machine learning benchmark examples.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fukumizu, K., Gretton, A., Sun, X., Schölkopf, B.: Kernel measures of conditional dependence. In: Advances in Neural Information Processing Systems 20, pp. 489–496 (2007)

    Google Scholar 

  2. Shawe-Taylor, J., Cristianini, N.: Kernel methods for pattern analysis. Cambridge University Press, New York (2004)

    Book  MATH  Google Scholar 

  3. Chwialkowski, K., Gretton, A.: A kernel independence test of random process. In: Proceedings of the 31th International Conference on Machine Learning, Beijing, China, pp. 1422–1430 (2014)

    Google Scholar 

  4. Gretton, A., Bousquet, O., Smola, A., Schölkopf, B.: Measuring statistical dependence with Hilbert-Schmidt norms. In: Proceedings of the 16th International Conference on Algorithmic Learning Theory, Singapore, pp. 63–77 (2005)

    Google Scholar 

  5. Song, L., Smola, A., Gretton, A., Borgwardt, K.: A dependence maximization view of clustering. In: Proceedings of the 24th International Conference on Machine Learning, Corvallis, USA, pp. 823–830 (2007)

    Google Scholar 

  6. Song, L., Smola, A., Gretton, A., Bedo, J., Borgwardt, K.: Feature selection via dependence maximization. J. Mach. Learn. Res. 13, 1393–1434 (2012)

    MathSciNet  MATH  Google Scholar 

  7. Wang, T., Zhao, D., Tian, S.: An overview of kernel alignment and its applications. Artif. Intell. Rev. 43(2), 179–192 (2015)

    Article  Google Scholar 

  8. Keerthi, S.S.: Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Trans. Neural Netw. 13(5), 1225–1229 (2002)

    Article  Google Scholar 

  9. Gönen, M., Alpayın, E.: Multiple kernel learning algorithms. J. Mach. Learn. Res. 12, 2211–2226 (2011)

    MathSciNet  MATH  Google Scholar 

  10. Wang, T., Tian, S., Huang, H., Deng, D.: Learning by local kernel polarization. Neurocomputing 72(13–15), 3077–3084 (2009)

    Article  Google Scholar 

  11. Cristianini, N., Shawe-Taylor, J., Elisseeff, A., Kandola, J.: On kernel-target alignment. In: Advances in Neural Information Processing Systems 14, pp 367–373 (2001)

    Google Scholar 

  12. Cortes, C., Mohri, M., Rostamizadeh, A.: Algorithms for learning kernels based on centered alignment. J. Mach. Learn. Res. 13, 795–828 (2012)

    MathSciNet  MATH  Google Scholar 

  13. Steinwart, I.: On the influence of the kernels on the consistency of support vector machines. J. Mach. Learn. Res. 2, 67–93 (2001)

    MathSciNet  MATH  Google Scholar 

  14. Sugiyama, M.: On kernel parameter selection in Hilbert-Schmidt independence criterion. IEICE Trans. Inf. Syst. E95-D(10), 2564–2567 (2012)

    Article  Google Scholar 

  15. Lu, Y., Wang, L., Lu, J., Yang, J., Shen, C.: Multiple kernel clustering based on centered kernel alignment. Pattern Recogn. 47(11), 3656–3664 (2014)

    Article  Google Scholar 

  16. Neumann, J., Schnörr, C., Steidl, G.: Combined SVM-based feature selection and classification. Mach. Learn. 61(1–3), 129–150 (2005)

    Article  MATH  Google Scholar 

  17. Lichman, M.: UCI machine learning repository. Irvine, CA: University of California, School of Information and Computer Science (2013). http://archive.ics.uci.edu/ml/

  18. Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), Article No. 27 (2011). http://www.csie.ntu.edu.tw/~cjlin/libsvm

    Google Scholar 

  19. Demšar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)

    MathSciNet  MATH  Google Scholar 

  20. Lu, J., Behbood, V., Hao, P., Zuo, H., Xue, S., Zhang, G.: Transfer learning using computational intelligence: a survey. Knowl.-Based Syst. 80, 14–23 (2015)

    Article  Google Scholar 

Download references

Acknowledgements

This work is supported in part by the National Natural Science Foundation of China (No. 61562003) and the Natural Science Foundation of Jiangxi Province of China (Nos. 20151BAB207029 and 20161BAB202070).

Author information

Authors and Affiliations

  1. School of Mathematics and Computer Science, Gannan Normal University, Ganzhou, 341000, P.R. China

    Tinghua Wang, Wei Li & Xianwen He

Authors
  1. Tinghua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xianwen He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tinghua Wang .

Editor information

Editors and Affiliations

  1. Institute of Automation, Chinese Academy of Sciences, Beijing, China

    Tieniu Tan

  2. Xi’an Institute of Optics and Precision Mechanics, Chinese Academy of Sciences, Xi'an, China

    Xuelong Li

  3. Chinese Academy of Sciences, Institute of Computing Technology, Beijing, China

    Xilin Chen

  4. Tsinghua University , Beijing, China

    Jie Zhou

  5. Nanjing University of Science and Technology, Nanjing, China

    Jian Yang

  6. University of Electronic Science and Technology, Chengdu, Sichuan, China

    Hong Cheng

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer Nature Singapore Pte Ltd.

About this paper

Cite this paper

Wang, T., Li, W., He, X. (2016). Kernel Learning with Hilbert-Schmidt Independence Criterion. In: Tan, T., Li, X., Chen, X., Zhou, J., Yang, J., Cheng, H. (eds) Pattern Recognition. CCPR 2016. Communications in Computer and Information Science, vol 662. Springer, Singapore. https://doi.org/10.1007/978-981-10-3002-4_58

Download citation

  • DOI: https://doi.org/10.1007/978-981-10-3002-4_58

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-10-3001-7

  • Online ISBN: 978-981-10-3002-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation