Abstract
This paper discusses an alternative approach to parameter optimization of prototype-based learning algorithms that aim to minimize an objective function based on gradient search. The proposed approach is a stochastic optimization method called the Cross Entropy (CE) method. The CE method is used to tackle the initialization sensitiveness problem associated with the original generalized Learning Vector Quantization (GLVQ) algorithm and its variants and to locate the globally optimal solutions. We will focus our study on a variant which deals with a weighted norm instead of the Euclidean norm in order to select the most relevant features. The results in this paper indicate that the CE method can successfully be applied to this kind of problems and efficiently generate high quality solutions. Also, highly competitive numerical results on real world data sets are reported.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Kohonen, T.: Learning vector quantization for pattern recognition. Technical Report TKK-F-A601 (1986)
Kohones, T.: Bibliography on the self-organizing map (som) and learning vector quantization(lvq) (2002)
Bojer, T., Hammer, B., Schunk, D., Toschanowitz, K.V.: Releavance determination in learning vector quantization. In: The European Symposium on Artificial Neural networks, pp. 271–276 (2001)
Hammer, B., Villman, T.: Estimating relevant input dimensions for self-organizing algorithms. Advances in Self-Organizing Maps, 173–180 (2001)
Sato, A.S., Yamada, K.: A formulation of learning vector quantization using a new misclassification measure. In: The 14th International Conference on Pattern Recognition (1998)
Hammer, B., Strickert, M., Villmann, T.: Relevance lvq versus svm. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 592–597. Springer, Heidelberg (2004)
Hammer, B., Strickert, M., Villmann, T.: Supervised neural gas with general similarity measure. Neural Processing Letters (2004)
Rubinstein, R.Y.: Optimization of computer simulation models with rare events. European Journal of operational Research 99, 89–112 (1997)
Diamantini, C., Spalvieri, A.: Certain facts about kohonen’s lvq1 algorithm. IEEE Transactions on Circuits and Systems I(47), 425–427 (1996)
Juang, B.H., Katagiri, S.: Discriminative learning for minimum error classification. IEEE Transactions on Signal Processing 40(2), 3043–3054 (1992)
Fu, M.C., Glover, F.W., April, J.: Simulation optimization: A review, new developements, and applications. In: WSC2005. The 37th Winter Simulation Conference, pp. 83–95 (2005)
Kroese, D., Porotsky, S., Rubinstein, R.: The cross-entropy method for continuous multi-extremal optimization. Methodology and Computing in Applied Probability 8, 383–407 (2006)
Rubinstein, R.Y., Kroese, D.P.: The Cross-Entropy Method: A Unified Approach to Combinatorial Method, Monte-Carlo Simulation, Randomized Optimization and Machine Learning. Springer, Heidelberg (2004)
Boer, P.D., Kroese, D., Mannor, S., Rubinstein, R.Y.: A tutorial on the cross-entropy method. Annals of Operations Research, 19–67 (2005)
Minka, T.: Estimating a dirichlet distribution (2003)
Newman, D.J., Hettich, S., Blake, C.L., Merz, C.J.: Uci repository of machine learning databases (1998)
Wu, J., Chung, A.: Cross entropy: A new solver for markov random field modeling and applications to medical image segmentation. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3749, pp. 229–237. Springer, Heidelberg (2005)
Qin, A., Suganthan, P.: Initialization insensitive lvq algorithm based on cost-function adaptation. Pattern Recognition 38(5), 773–776 (2005)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2007 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Boubezoul, A., Paris, S., Ouladsine, M. (2007). Improving the GRLVQ Algorithm by the Cross Entropy Method. In: de Sá, J.M., Alexandre, L.A., Duch, W., Mandic, D. (eds) Artificial Neural Networks – ICANN 2007. ICANN 2007. Lecture Notes in Computer Science, vol 4668. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-74690-4_21
Download citation
DOI: https://doi.org/10.1007/978-3-540-74690-4_21
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-74689-8
Online ISBN: 978-3-540-74690-4
eBook Packages: Computer ScienceComputer Science (R0)