Skip to main content
SpringerLink
Log in
Book cover

Multi-Objective Machine Learning pp 199–220Cite as

Multi-Objective Optimization of Support Vector Machines

  • Chapter

Part of the book series: Studies in Computational Intelligence ((SCI,volume 16))

Abstract

Designing supervised learning systems is in general a multi-objective optimization problem. It requires finding appropriate trade-offs between several objectives, for example between model complexity and accuracy or sensitivity and specificity. We consider the adaptation of kernel and regularization parameters of support vector machines (SVMs) by means of multi-objective evolutionary optimization. Support vector machines are reviewed from the multi-objective perspective, and different encodings and model selection criteria are described. The optimization of split modified radius-margin model selection criteria is demonstrated on benchmark problems. The MOO approach to SVM design is evaluated on a real-world pattern recognition task, namely the real-time detection of pedestrians in infrared images for driver assistance systems. Here the three objectives are the minimization of the false positive rate, the false negative rate, and the number of support vectors to reduce the computational complexity.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   169.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD   219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Hussein A. Abbass. An evolutionary artificial neural networks approach for breast cancer diagnosis. Artificial Intelligence in Medicine, 25(3):265–281, 2002.

    Article  Google Scholar 

  2. Hussein A. Abbass. Speeding up backpropagation using multiobjective evolutionary algorithms. Neural Computation, 15(11):2705–2726, 2003.

    Article  MATH  Google Scholar 

  3. Massimo Bertozzi, Alberto Broggi, Marcello Carletti, Alessandra Fascioli, Thorsten Graf, Paolo Grisleri, and Michael Meinecke. IR pedestrian detection for advanced driver assistance systems. In Proceedings of the 25th Pattern Recognition Symposium, volume 2781 of LNCS, pages 582–590. Springer-Verlag, 2003.

    Google Scholar 

  4. Massimo Bertozzi, Alberto Broggi, Alessandra Fascioli, and Massimiliano Sechi. Shape-based pedestrian detection. In Proceedings of the IEEE Intelligent Vehicles Symposium 2000, pages 215–220, 2000.

    Google Scholar 

  5. Massimo Bertozzi, Alberto Broggi, Thorsten Graf, Paolo Grisleri, and Michael Meinecke. Pedestrian detection in infrared images. In Proceedings of the IEEE Intelligent Vehicles Symposium 2003, pages 662–667, 2003.

    Google Scholar 

  6. J. Bi. Multi-objective programming in SVMs. In Tom Fawcett and Nina Mishra, editors, Machine Learning, Proceedings of the 20th International Conference (ICML 2003), pages 35–42. AAAI Press, 2003.

    Google Scholar 

  7. C.L. Blake and C.J. Merz. UCI repository of machine learning databases, 1998.

    Google Scholar 

  8. J Canny. A computational approach to edge detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 8(6):679–698, 1986.

    Article  Google Scholar 

  9. O. Chapelle, V. Vapnik, O. Bousquet, and S. Mukherjee. Choosing multiple parameters for support vector machines. Machine Learning, 46(1):131–159, 2002.

    Article  MATH  Google Scholar 

  10. K.-M. Chung, W.-C. Kao, C.-L. Sun, and C.-J. Lin. Radius margin bounds for support vector machines with RBF kernel. Neural Computation, 15(11):2643–2681, 2003.

    Article  MATH  Google Scholar 

  11. Carlos A. Coello Coello, David A. Van Veldhuizen, and Gary B. Lamont. Evolutionary Algorithms for Solving Multi-Objective Problems. Kluwer Academic Publishers, 2002.

    Google Scholar 

  12. Nello Cristianini and John Shawe-Taylor. An Introduction to Support Vector Machines and other kernel-based learning methods. Cambridge University Press, 2000.

    Google Scholar 

  13. C. Curio, J. Edelbrunner, T. Kalinke, C. Tzomakas, and W. von Seelen. Walking pedestrian recognition. IEEE Transactions on Intelligent Transportation Systems, 1(3):155–163, 2000.

    Article  Google Scholar 

  14. Kalyanmoy Deb. Multi-Objective Optimization Using Evolutionary Algorithms. Wiley, 2001.

    Google Scholar 

  15. Kalyanmoy Deb, Samir Agrawal, Amrit Pratap, and T. Meyarivan. A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Transactions on Evolutionary Computation, 6(2):182–197, 2002.

    Article  Google Scholar 

  16. K. Duan, S. S. Keerthi, and A.N. Poo. Evaluation of simple performance measures for tuning SVM hyperparameters. Neurocomputing, 51:41–59, 2003.

    Article  Google Scholar 

  17. Damian R. Eads, Daniel Hill, Sean Davis, Simon J. Perkins, Junshui Ma, Reid B. Porter, and James P. Theiler. Genetic algorithms and support vector machines for time series classification. In Bruno Bosacchi, David B. Fogel, and James C. Bezdek, editors, Applications and Science of Neural Networks, Fuzzy Systems, and Evolutionary Computation V., volume 4787 of Proceedings of the SPIE, pages 74–85, 2002.

    Google Scholar 

  18. C. M. Fonseca, J. D. Knowles, L. Thiele, and E. Zitzler. A tutorial on the performance assessment of stochastic multiobjective optimizers. Presented at the Third International Conference on Evolutionary Multi-Criterion Optimization (EMO 2005), 2005.

    Google Scholar 

  19. Frauke Friedrichs and Christian Igel. Evolutionary tuning of multiple SVM parameters. Neurocomputing, 64(C):107–117, 2005.

    Google Scholar 

  20. H. Fröhlich, O. Chapelle, and B. Schölkopf. Feature selection for support vector machines using genetic algorithms. International Journal on Artificial Intelligence Tools, 13(4):791–800, 2004.

    Article  Google Scholar 

  21. Tobias Glasmachers and Christian Igel. Gradient-based adaptation of general gaussian kernels. Neural Computation, 17(10):2099–2105, 2005.

    Article  MATH  MathSciNet  Google Scholar 

  22. Carl Gold and Peter Sollich. Model selection for support vector machine classification. Neurocomputing, 55(1–2):221–249, 2003.

    Article  Google Scholar 

  23. Nikolaus Hansen. Invariance, self-adaptation and correlated mutations and evolution strategies. In Proceedings of the 6th International Conference on Parallel Problem Solving from Nature (PPSN VI), volume 1917 of LNCS, pages 355–364. Springer-Verlag, 2000.

    Google Scholar 

  24. Trevor Hastie, Robert Tibshirani, and Jerome Friedman. The Elements of Statistical Learning Data Mining, Inference, and Prediction. Springer Series in Statistics. Springer-Verlag, 2001.

    Google Scholar 

  25. C. Igel. Multi-objective model selection for support vector machines. In C. A. Coello Coello, E. Zitzler, and A. Hernandez Aguirre, editors, Proceedings of the Third International Conference on Evolutionary Multi-Criterion Optimization (EMO 2005), volume 3410 of LNCS, pages 534–546. Springer-Verlag, 2005.

    Google Scholar 

  26. Yaochu Jin, Tatsuya Okabe, and Bernhard Sendhoff. Neural network regularization and ensembling using multi-objective evolutionary algorithms. In Congress on Evolutionary Computation (CEC’04), pages 1–8. IEEE Press, 2004.

    Google Scholar 

  27. Kees Jong, Elena Marchiori, and Aad van der Vaart. Analysis of proteomic pattern data for cancer detection. In G. R. Raidl, S. Cagnoni, J. Branke, D. W. Corne, R. Drechsler, Y. Jin, C. G. Johnson, P. Machado, E. Marchiori, F. Rothlauf, G. D. Smith, and G. Squillero, editors, Applications of Evolutionary Computing, volume 3005 of LNCS, pages 41–51. Springer-Verlag, 2004.

    Google Scholar 

  28. S. S. Keerthi. Efficient tuning of SVM hyperparameters using radius/margin bound and iterative algorithms. IEEE Transactions on Neural Networks, 13(5):1225–1229, 2002.

    Article  Google Scholar 

  29. M. T. Miller, A. K. Jerebko, J. D. Malley, and R. M. Summers. Feature selection for computer-aided polyp detection using genetic algorithms. In Anne V. Clough and Amir A. Amini, editors, Medical Imaging 2003: Physiology and Function: Methods, Systems, and Applications, volume 5031 of Proceedings of the SPIE, pages 102–110, 2003.

    Google Scholar 

  30. Anuj Mohan, Constantine Papageorgiou, and Thomas Poggio. Example-based object detection in images by components. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(4):349–361, 2001.

    Article  Google Scholar 

  31. Katharina Morik, Peter Brockhausen, and Thorsten Joachims. Combining statistical learning with a knowledge-based approach - a case study in intensive care monitoring. In Proceedings of the 16th International Conference on Machine Learning, pages 268–277. Morgan Kaufmann, 1999.

    Google Scholar 

  32. M. Oren, C. P. Papageorgiou, P. Sinha, E. Osuna, and T. Poggio. Pedestrian detection using wavelet templates. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 193–199, 1997.

    Google Scholar 

  33. S. Pang and N. Kasabov. Inductive vs. transductive inference, global vs. local models: SVM, TSVM, and SVMT for gene expression classification problems. In International Joint Conference on Neual Networks (IJCNN), volume 2, pages 1197–1202. IEEE Press, 2004.

    Google Scholar 

  34. C. Papageorgiou, T. Evgeniou, and T. Poggio. A trainable pedestrian detection system. In Proceedings of the IEEE International Conference on Intelligent Vehicles Symposium 1998, pages 241–246, 1998.

    Google Scholar 

  35. C. P. Papageorgiou. A trainable system for object detection in images and video sequences. Technical Report AITR-1685, Massachusetts Institute of Technology, Artificial Intelligene Laboratory, 2000.

    Google Scholar 

  36. Constantine Papageorgiou and Tomaso Poggio. A trainable system for object detection. International Journal of Computer Vision, 38(1):15–33, 2000.

    Article  MATH  Google Scholar 

  37. G. Rätsch, T. Onoda, and K.-R. Müller. Soft margins for AdaBoost. Machine Learning, 42(3):287–320, 2001.

    Article  MATH  Google Scholar 

  38. Günther Rudolph. On correlated mutations in evolution strategies. In R. Männer and B. Manderick, editors, Parallel Problem Solving from Nature 2 (PPSN II), pages 105–114. Elsevier, 1992.

    Google Scholar 

  39. Thomas Philip Runarsson and Sven Sigurdsson. Asynchronous parallel evolutionary model selection for support vector machines. Neural Information Processing - Letters and Reviews, 3(3):59–68, 2004.

    Google Scholar 

  40. Y. Sawaragi, H. Nakayama, and T. Tanino. Theory of Multiobjective Optimization, volume 176 of Mathematics in Science and Engineering. Academic Press, 1985.

    Google Scholar 

  41. B. Schölkopf, C. J. C. Burges, and V. Vapnik. Extracting support data for a given task. In U. M. Fayyad and R. Uthurusamy, editors, Proceedings of the First International Conference on Knowledge Discovery & Data Mining, pages 252–257, Menlo Park, CA, 1995. AAAI Press.

    Google Scholar 

  42. B. Schölkopf and A. J. Smola. Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond. MIT Press, 2002.

    Google Scholar 

  43. John Shawe-Taylor and Nello Cristianini. Kernel Methods for Pattern Analysis. Cambridge University Press, 2004.

    Google Scholar 

  44. S. Y. M. Shi, P. N. Suganthan, and K. Deb. Multi-class protein fold recognition using multi-objective evolutionary algorithms. In IEEE Symposium on Computational Intelligence in Bioinformatics and Computational Biology, pages 61–66, 2004.

    Google Scholar 

  45. Vladimir N. Vapnik. The Nature of Statistical Learning Theory. Springer- Verlag, 1995.

    Google Scholar 

  46. S. Wiegand, C. Igel, and U. Handmann. Evolutionary multi-objective optimization of neural networks for face detection. International Journal of Computational Intelligence and Applications, 4(3):237–253, 2004. Special issue on Neurocomputing and Hybrid Methods for Evolving Intelligence.

    Article  Google Scholar 

  47. F. Xu, X. Liu, and K. Fujimura. Pedestrian detection and tracking with night vision. IEEE Transactions on Intelligent Transportation Systems, 6(1):63–71, 2005.

    Article  Google Scholar 

  48. L. Zhao and C. Thorpe. Stereo- and neural network-based pedestrian detection. In Proceedings of the IEEE International Conference on Intelligent Transportation Systems’99, pages 298–303, 1999.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institut für Neuroinformatik Ruhr-Universität Bochum, Bochum, 44780, Germany

    Thorsten Suttorp & Christian Igel

Authors
  1. Thorsten Suttorp
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Christian Igel
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Honda Research Institute Europe GmbH, Carl-Legien-Str. 30, Offenbach, 63073, Germany

    Yaochu Jin Dr.

Rights and permissions

Reprints and permissions

Copyright information

© 2006 Springer

About this chapter

Cite this chapter

Suttorp, T., Igel, C. (2006). Multi-Objective Optimization of Support Vector Machines. In: Jin, Y. (eds) Multi-Objective Machine Learning. Studies in Computational Intelligence, vol 16. Springer, Berlin, Heidelberg. https://doi.org/10.1007/3-540-33019-4_9

Download citation

  • DOI: https://doi.org/10.1007/3-540-33019-4_9

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-30676-4

  • Online ISBN: 978-3-540-33019-6

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation