Skip to main content

Advertisement

Log in

Genetic algorithm–based training for semi-supervised SVM

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

The Support Vector Machine (SVM) is an interesting classifier with excellent power of generalization. In this paper, we consider applying the SVM to semi-supervised learning. We propose using an additional criterion with the standard formulation of the semi-supervised SVM (S 3 VM) to reinforce classifier regularization. Since, we deal with nonconvex and combinatorial problem, we use a genetic algorithm to optimize the objective function. Furthermore, we design the specific genetic operators and certain heuristics in order to improve the optimization task. We tested our algorithm on both artificial and real data and found that it gives promising results in comparison with classical optimization techniques proposed in literature.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

Notes

  1. The problem described in Fig. 2 is just a visual example of the introduction of the regularization idea. In addition, it is important to state that we use the same kernel in Fig. 2a, b. Also, we have repeated the example with other kernels with the same result.

References

  1. Adankon MM, Cheriet M (2007) Learning semi-supervised svm with genetic algorithm. In: IEEE, editor, International joint conference in neural networks 2007, orlando, FL, pp 1825–1830

  2. Adankon MM, Cheriet M (2007) Optimizing resources in model selection for support vector machines. Pattern Recognit 40(3):953–963

    Article  Google Scholar 

  3. Back T (1996) Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford University Press, New York

    Google Scholar 

  4. Back T, Hammel U, Schwefel HP (1997) Evolutionary computation: comments on the history and current state. IEEE Trans Evol Comput 1:3–17

    Article  Google Scholar 

  5. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434

    MathSciNet  Google Scholar 

  6. Bennett K, Demiriz A (1998) Semi-supervised support vector machines

  7. Camps-Valls G, Bandos TV, Zhou D (2007) Semi-supervised graph-based hyperspectral image classification. IEEE Trans Geosci Remote Sens 45(10):3044–3054

    Article  Google Scholar 

  8. Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MA

    Google Scholar 

  9. Chapelle O, Sindhwani V, Keerthi SS (2006) Branch and bound for semi-supervised support vector machines. In: Proceedings of the NIPS 2006

  10. Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233

    Google Scholar 

  11. Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1):131–159

    Article  Google Scholar 

  12. Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the tenth international workshop on artificial intelligence and statistics

  13. Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712

    MathSciNet  Google Scholar 

  14. Demiriz A, Bennett K, Embrechts M (1999) Semi-supervised clustering using genetic algorithms. Technical report, R.P.I. Math Report No. 9901, Rensselaer Polytechnic Institute, Troy, New York

  15. Fröhlich H, Schölkopf B, Chapelle O (2003) Feature selection for support vector machines using genetic algorithms. In: International journal on artificial intelligence tools, pp 142–148. IEEE Computer Society

  16. Friedrichs F, Igel C (2005) Evolutionary tuning of multiple svm parameters. In: Trends in neurocomputing: 12th European symposium on artificial neural networks 2004, vol 64, pp 107–117

  17. Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Massachusetts

    Google Scholar 

  18. Grandvalet Y, Bengio Y (2004) Semi-supervised learning by entropy minimization. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, Cambridge, MA, pp 529–536

    Google Scholar 

  19. Hein M, Audibert J-Y, Von Luxburg U (2007) Graph laplacians and their convergence on random neighborhood graphs. J Mach Learn Res 8:1325–1370

    MathSciNet  Google Scholar 

  20. Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Michigan

    Google Scholar 

  21. Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence, 2nd edn. MIT Press, Cambridge, MA

    Google Scholar 

  22. Joachims T (1999) Transductive inference for text classification using support vector machines. In: Bratko I, Dzeroski S (eds) Proceedings of ICML-99, 16th international conference on machine learning. Bled, SL. Morgan Kaufmann Publishers, San Francisco, US, pp 200–209

  23. Lorena AC, de Carvalho ACPLF (2008) Evolutionary tuning of svm parameter values in multiclass problems. Neurocomputing 71(16–18):3326–3334

    Article  Google Scholar 

  24. Mitchell TM (1999) The role of unlabeled data in supervised learning. In: Proceedings of the sixth international colloquium on cognitive science, San Sebastian, Spain

  25. Bonissone PP, Subbu R, Eklund N, Kiehl TR (2006) Evolutionary algorithms + domain knowledge = real-world evolutionary computation. IEEE Trans Evol Comput 10(3):256–280

    Article  Google Scholar 

  26. Scholkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge, MA

    Google Scholar 

  27. Seeger M (2001) Learning with labeled and unlabeled data. Technical report, Technical Report, Institute for adaptive and neural computation, University of Edinburgh

  28. Silva MM, Maia TT, Braga AP (2005) An evolutionary approach to transduction in support vector machines. In: Proceedings of the fifth international conference on hybrid intelligent systems, pp 329–334

  29. Spears WM (1998) The role of mutation and recombinaison in evolutionary algorithms. Ph.D. Thesis, George Mason University, Fairfax, VA

  30. Szummer M, Jaakkola T (2001) Partially labeled classification with markov random walks. In: Advances in neural information processing systems, vol 14

  31. Vapnik VN (1998) Statistical learning theory. Wiley, New York

    Google Scholar 

  32. Zhou D, Bousquet O, Lal T, Weston J, Schölkopf B (2003) Learning with local and global consistency. In: 18th Annual conference on neural information processing systems

  33. Zhu X (2008) Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison. http://www.cs.wisc.edu/∼jerryzhu/pub/ssl_survey.pdf

Download references

Acknowledgments

The authors would like to thank the NSERC of Canada for their financial support.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mathias M. Adankon.

Additional information

This work is extended version of the IJCNN2007 paper, where we include the following important new information: a thorough explanation and analysis of the objective function, use of the connectivity concept for selecting the unlabeled subset during the optimization process and improved experimental results.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Adankon, M.M., Cheriet, M. Genetic algorithm–based training for semi-supervised SVM. Neural Comput & Applic 19, 1197–1206 (2010). https://doi.org/10.1007/s00521-010-0358-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-010-0358-8

Keywords

Navigation