Skip to main content
SpringerLink
Log in

Kernel Construction and Feature Subset Selection in Support Vector Machines

  • Conference paper
  • First Online:
Simulated Evolution and Learning (SEAL 2017)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 10593))

Included in the following conference series:

  • 3087 Accesses

Abstract

Kernel functions have an important role in the performance of Support Vector Machines (SVMs), since they form the geometry of the feature space. Manual designing of kernel functions is an expensive task and requires domain-specific knowledge. In this article, we propose a new method to automatically construct kernel functions and select optimal subsets of features. We achieve this by combining primitive kernels and subsets of features using Genetic Programming (GP). Our experiments show that the proposed method drastically improves the prediction accuracy of SVMs.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bach, F.R., Lanckriet, G.R.G., Jordan, M.I.: Multiple kernel learning, conic duality, and the SMO algorithm. In: Proceedings of 21st International Conference on Machine Learning, ICML 2004, p. 6. ACM, New York (2004)

    Google Scholar 

  2. Bartz-Beielstein, T., Lasarczyk, C., Preuss, M.: The sequential parameter optimization toolbox. In: Bartz-Beielstein, T., Chiarandini, M., Paquete, L., Preuss, M. (eds.) Experimental Methods for the Analysis of Optimization Algorithms, pp. 337–360. Springer, New York (2010). doi:10.1007/978-3-642-02538-9_14

    Chapter  Google Scholar 

  3. Bhowan, U., McCloskey, D.J.: Genetic programming for feature selection and question-answer ranking in IBM Watson. In: Machado, P., Heywood, M.I., McDermott, J., Castelli, M., García-Sánchez, P., Burelli, P., Risi, S., Sim, K. (eds.) EuroGP 2015. LNCS, vol. 9025, pp. 153–166. Springer, Cham (2015). doi:10.1007/978-3-319-16501-1_13

    Google Scholar 

  4. Chang, C.-C., Lin, C.-J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. 2(3), 27:1–27:27 (2011)

    Article  Google Scholar 

  5. Cortes, C., Haffner, P., Mohri, M., Bennett, K., Cesa-bianchi, N.: Rational kernels. J. Mach. Learn. Res. 5, 1035–1062 (2004)

    MathSciNet  MATH  Google Scholar 

  6. Diosan, L., Rogozan, A., Pecuchet, J.P.: Evolving kernel functions for SVMs by genetic programming. In: Sixth International Conference on Machine Learning and Applications (ICMLA 2007), pp. 19–24, December 2007

    Google Scholar 

  7. Dioşan, L., Rogozan, A., Pecuchet, J.-P.: Improving classification performance of support vector machine by genetically optimising kernel shape and hyper-parameters. Appl. Intell. 36(2), 280–294 (2012)

    Article  Google Scholar 

  8. Gray, H.F., Maxwell, R.J., Martnez-Prez, I., Ars, C., Cerdn, S.: Genetic programming for classication and feature selection: analysis of 1h nuclear magnetic resonance spectra from human brain tumour biopsies. NMR Biomed. 11(4–5), 217–224 (1998)

    Article  Google Scholar 

  9. Howley, T., Madden, M.G.: The genetic kernel support vector machine: description and evaluation. Artif. Intell. Rev. 24(3), 379–395 (2005)

    Article  Google Scholar 

  10. Kloft, M., Brefeld, U., Sonnenburg, S., Zien, A.: Lp-norm multiple kernel learning. J. Mach. Learn. Res. 12, 953–997 (2011)

    MathSciNet  MATH  Google Scholar 

  11. Koch, P., Bischl, B., Flasch, O., Bartz-Beielstein, T., Weihs, C., Konen, W.: Tuning and evolution of support vector kernels. Evol. Intel. 5(3), 153–170 (2012)

    Article  Google Scholar 

  12. Konen, W., Koch, P., Flasch, O., Bartz-Beielstein, T., Friese, M., Naujoks, B.: Tuned data mining: a benchmark study on different tuners. In: Proceedings of 13th Annual Conference on Genetic and Evolutionary Computation, GECCO 2011, pp. 1995–2002. ACM, New York (2011)

    Google Scholar 

  13. Koza, J.R.: Genetic Programming: On the Programming of Computers by Means of Natural Selection. MIT Press, Cambridge (1992)

    MATH  Google Scholar 

  14. Lanckriet, G.R., Cristianini, N., Bartlett, P., Ghaoui, L.E., Jordan, M.I.: Learning the kernel matrix with semidefinite programming. J. Mach. Learn. Res. 5, 27–72 (2004)

    MathSciNet  MATH  Google Scholar 

  15. Lichman, M.: UCI machine learning repository (2013)

    Google Scholar 

  16. Meenakshi, A.R., Rajian, C.: On a product of positive semidefinite matrices. Linear Algebra Appl. 295(1), 3–6 (1999)

    Article  MathSciNet  MATH  Google Scholar 

  17. Micchelli, C.A., Pontil, M.: Learning the kernel function via regularization. J. Mach. Learn. Res. 6, 1099–1125 (2005)

    MathSciNet  MATH  Google Scholar 

  18. Orabona, F., Fornoni, M., Caputo, B., Cesa-Bianchi, N.: OM-2: an online multi-class multi-kernel learning algorithm Luo Jie. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR Workshops 2010, San Francisco, CA, USA, 13–18 June 2010, pp. 43–50. IEEE Computer Society (2010)

    Google Scholar 

  19. Orabona, F., Luo, J.: Ultra-fast optimization algorithm for sparse multi kernel learning. In: Proceedings of 28th International Conference on Machine Learning, no. Idiap-RR-11-2011, June 2011

    Google Scholar 

  20. Orabona, F., Luo, J., Caputo, B.: Online-batch strongly convex multi kernel learning. In: The 23rd IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2010, San Francisco, CA, USA, 13–18 June 2010, pp. 787–794. IEEE Computer Society (2010)

    Google Scholar 

  21. Silva, S., Almeida, J.: GPLAB-a genetic programming toolbox for MATLAB. In: Proceedings of Nordic MATLAB Conference (NMC-2003), pp. 273–278 (2005)

    Google Scholar 

  22. Steinwart, I., Christmann, A.: Support Vector Machines, 1st edn. Springer Publishing Company Incorporated, New York (2008)

    MATH  Google Scholar 

  23. Sullivan, K.M., Luke, S.: Evolving kernels for support vector machine classification. In: Thierens, D., Beyer, H.-G., Bongard, J., Branke, J., Clark, J.A., Cliff, D., Congdon, C.B., Deb, K., Doerr, B., Kovacs, T., Kumar, S., Miller, J.F., Moore, J., Neumann, F., Pelikan, M., Poli, R., Sastry, K., Stanley, K.O., Stutzle, T., Watson, R.A., Wegener, I. (eds.) GECCO 2007: Proceedings of 9th Annual Conference on Genetic and Evolutionary Computation, vol. 2, pp. 1702–1707. ACM Press, London, 7–11 July 2007

    Google Scholar 

  24. Surez, R.R., Valencia-Ramrez, J.M., Graff, M.: Genetic programming as a feature selection algorithm. In: 2014 IEEE International Autumn Meeting on Power, Electronics and Computing (ROPEC), pp. 1–5, November 2014

    Google Scholar 

  25. Xue, B., Zhang, M., Browne, W.N., Yao, X.: A survey on evolutionary computation approaches to feature selection. IEEE Trans. Evol. Comput. 1(99), 1 (2015)

    Google Scholar 

  26. Yamada, S., Neshatian, K., Sainudiin, R.: Optimal hyper-parameter search in support vector machines using Bézier surfaces. In: Pfahringer, B., Renz, J. (eds.) AI 2015. LNCS, vol. 9457, pp. 623–629. Springer, Cham (2015). doi:10.1007/978-3-319-26350-2_55

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science and Software Engineering, University of Canterbury, Christchurch, New Zealand

    Shinichi Yamada & Kourosh Neshatian

Authors
  1. Shinichi Yamada
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kourosh Neshatian
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shinichi Yamada .

Editor information

Editors and Affiliations

  1. Southern University of Science and Technology, Shenzhen, China

    Yuhui Shi

  2. City University of Hong Kong, Hong Kong, Kowloon, Hong Kong

    Kay Chen Tan

  3. Victoria University of Wellington, Wellington, Wellington, New Zealand

    Mengjie Zhang

  4. Southern University of Science and Technology, Shenzhen, China

    Ke Tang

  5. RMIT University, Melbourne, Victoria, Australia

    Xiaodong Li

  6. City University of Hong Kong, Kowloon Tong, Hong Kong

    Qingfu Zhang

  7. Peking University, Beijing, China

    Ying Tan

  8. University of Leipzig, Leipzig, Germany

    Martin Middendorf

  9. University of Surrey, Guildford, Surrey, United Kingdom

    Yaochu Jin

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this paper

Cite this paper

Yamada, S., Neshatian, K. (2017). Kernel Construction and Feature Subset Selection in Support Vector Machines. In: Shi, Y., et al. Simulated Evolution and Learning. SEAL 2017. Lecture Notes in Computer Science(), vol 10593. Springer, Cham. https://doi.org/10.1007/978-3-319-68759-9_49

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-68759-9_49

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-68758-2

  • Online ISBN: 978-3-319-68759-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation