Abstract
The Support Vector Machine (SVM) is an interesting classifier with excellent power of generalization. In this paper, we consider applying the SVM to semi-supervised learning. We propose using an additional criterion with the standard formulation of the semi-supervised SVM (S 3 VM) to reinforce classifier regularization. Since, we deal with nonconvex and combinatorial problem, we use a genetic algorithm to optimize the objective function. Furthermore, we design the specific genetic operators and certain heuristics in order to improve the optimization task. We tested our algorithm on both artificial and real data and found that it gives promising results in comparison with classical optimization techniques proposed in literature.
Similar content being viewed by others
References
Adankon MM, Cheriet M (2007) Learning semi-supervised svm with genetic algorithm. In: IEEE, editor, International joint conference in neural networks 2007, orlando, FL, pp 1825–1830
Adankon MM, Cheriet M (2007) Optimizing resources in model selection for support vector machines. Pattern Recognit 40(3):953–963
Back T (1996) Evolutionary algorithms in theory and practice: evolution strategies, evolutionary programming, genetic algorithms. Oxford University Press, New York
Back T, Hammel U, Schwefel HP (1997) Evolutionary computation: comments on the history and current state. IEEE Trans Evol Comput 1:3–17
Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: a geometric framework for learning from labeled and unlabeled examples. J Mach Learn Res 7:2399–2434
Bennett K, Demiriz A (1998) Semi-supervised support vector machines
Camps-Valls G, Bandos TV, Zhou D (2007) Semi-supervised graph-based hyperspectral image classification. IEEE Trans Geosci Remote Sens 45(10):3044–3054
Chapelle O, Schölkopf B, Zien A (eds) (2006) Semi-supervised learning. MIT Press, Cambridge, MA
Chapelle O, Sindhwani V, Keerthi SS (2006) Branch and bound for semi-supervised support vector machines. In: Proceedings of the NIPS 2006
Chapelle O, Sindhwani V, Keerthi SS (2008) Optimization techniques for semi-supervised support vector machines. J Mach Learn Res 9:203–233
Chapelle O, Vapnik V, Bousquet O, Mukherjee S (2002) Choosing multiple parameters for support vector machines. Mach Learn 46(1):131–159
Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. In: Proceedings of the tenth international workshop on artificial intelligence and statistics
Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712
Demiriz A, Bennett K, Embrechts M (1999) Semi-supervised clustering using genetic algorithms. Technical report, R.P.I. Math Report No. 9901, Rensselaer Polytechnic Institute, Troy, New York
Fröhlich H, Schölkopf B, Chapelle O (2003) Feature selection for support vector machines using genetic algorithms. In: International journal on artificial intelligence tools, pp 142–148. IEEE Computer Society
Friedrichs F, Igel C (2005) Evolutionary tuning of multiple svm parameters. In: Trends in neurocomputing: 12th European symposium on artificial neural networks 2004, vol 64, pp 107–117
Goldberg DE (1989) Genetic algorithms in search, optimization and machine learning. Addison-Wesley, Massachusetts
Grandvalet Y, Bengio Y (2004) Semi-supervised learning by entropy minimization. In: Saul LK, Weiss Y, Bottou L (eds) Advances in neural information processing systems 17. MIT Press, Cambridge, MA, pp 529–536
Hein M, Audibert J-Y, Von Luxburg U (2007) Graph laplacians and their convergence on random neighborhood graphs. J Mach Learn Res 8:1325–1370
Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence. University of Michigan Press, Michigan
Holland JH (1992) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence, 2nd edn. MIT Press, Cambridge, MA
Joachims T (1999) Transductive inference for text classification using support vector machines. In: Bratko I, Dzeroski S (eds) Proceedings of ICML-99, 16th international conference on machine learning. Bled, SL. Morgan Kaufmann Publishers, San Francisco, US, pp 200–209
Lorena AC, de Carvalho ACPLF (2008) Evolutionary tuning of svm parameter values in multiclass problems. Neurocomputing 71(16–18):3326–3334
Mitchell TM (1999) The role of unlabeled data in supervised learning. In: Proceedings of the sixth international colloquium on cognitive science, San Sebastian, Spain
Bonissone PP, Subbu R, Eklund N, Kiehl TR (2006) Evolutionary algorithms + domain knowledge = real-world evolutionary computation. IEEE Trans Evol Comput 10(3):256–280
Scholkopf B, Smola AJ (2002) Learning with kernels. MIT Press, Cambridge, MA
Seeger M (2001) Learning with labeled and unlabeled data. Technical report, Technical Report, Institute for adaptive and neural computation, University of Edinburgh
Silva MM, Maia TT, Braga AP (2005) An evolutionary approach to transduction in support vector machines. In: Proceedings of the fifth international conference on hybrid intelligent systems, pp 329–334
Spears WM (1998) The role of mutation and recombinaison in evolutionary algorithms. Ph.D. Thesis, George Mason University, Fairfax, VA
Szummer M, Jaakkola T (2001) Partially labeled classification with markov random walks. In: Advances in neural information processing systems, vol 14
Vapnik VN (1998) Statistical learning theory. Wiley, New York
Zhou D, Bousquet O, Lal T, Weston J, Schölkopf B (2003) Learning with local and global consistency. In: 18th Annual conference on neural information processing systems
Zhu X (2008) Semi-supervised learning literature survey. Technical Report 1530, Computer Sciences, University of Wisconsin-Madison. http://www.cs.wisc.edu/∼jerryzhu/pub/ssl_survey.pdf
Acknowledgments
The authors would like to thank the NSERC of Canada for their financial support.
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is extended version of the IJCNN2007 paper, where we include the following important new information: a thorough explanation and analysis of the objective function, use of the connectivity concept for selecting the unlabeled subset during the optimization process and improved experimental results.
Rights and permissions
About this article
Cite this article
Adankon, M.M., Cheriet, M. Genetic algorithm–based training for semi-supervised SVM. Neural Comput & Applic 19, 1197–1206 (2010). https://doi.org/10.1007/s00521-010-0358-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-010-0358-8