Abstract
In this paper we investigate possibilities of relevance learning in learning matrix quantization and discuss their mathematical properties. Learning matrix quantization can be seen as an extension of the learning vector quantization method, which is one of the most popular and intuitive prototype based vector quantization algorithms for classification learning. Whereas in the vector quantization approach vector data are processed, learning matrix quantization deals with matrix data as they occur in image processing of gray-scale images or in time-resolved spectral analysis. Here, we concentrate on the consideration of relevance learning when learning matrix quantization is based on the Schatten-p-norm as the data dissimilarity measure. For those matrix systems exist more relevance learning variants than for vector classification systems. We contemplate several approaches based on different matrix products as well as tensor operators. In particular, we discuss their mathematical properties related to the relevance learning task keeping in mind the stochastic gradient learning scheme for both, prototype as well as relevance learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Allen, G., Grosenick, L., Taylor, J.: A generalized least squares matrix decomposition. Journal of the American Statistical Association, Theory & Methods 109(505), 145–159 (2012)
Bengio, Y.: Learning deep architectures for AI. Foundations and Trends in Machine Learning 2(1), 1–127 (2009)
Biehl, M., Hammer, B., Schneider, P., Villmann, T.: Metric learning for prototype-based classification. In: Bianchini, M., Maggini, M., Scarselli, F., Jain, L. (eds.) Innovations in Neural Information Paradigms and Applications. SCI, vol. 247, pp. 183–199. Springer, Berlin (2009)
Bojer, T., Hammer, B., Schunk, D., von Toschanowitz, K.T.: Relevance determination in learning vector quantization. In: Proceedings of the 9th European Symposium on Artificial Neural Networks, ESANN 2001, D-Facto, Evere, Belgium, pp. 271–276 (2001)
Cichocki, A., Amari, S.: Adaptive Blind Signal and Image Processing. John Wiley (2002)
Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, A.: Margin analysis of the LVQ algorithm. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Advances in Neural Information Processing (Proc. NIPS 2002), vol. 15, pp. 462–469. MIT Press, Cambridge (2003)
Domaschke, K., Kaden, M., Lange, M., Villmann, T.: Learning matrix quantization and variants of relevance learning. In: Verleysen, M. (ed.) Proceedings of the European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2015), Louvain-La-Neuve, Belgium, page submitted (2015), i6doc.com
Duda, R., Hart, P.: Pattern Classification and Scene Analysis. Wiley, New York (1973)
Goldhorn, K.-H., Heinz, H.-P., Kraus, M.: Moderne mathematische Methoden der Physik, vol. 1. Springer, Heidelberg (2009)
Golub, G., Loan, C.V.: Matrix Computations, 4th edn. Johns Hopkins Studies in the Mathematical Sciences. John Hopkins University Press, Baltimore (2013)
Gu, Z., Shao, M., Li, L., Fu, Y.: Discriminative metric: Schatten norms vs. vector norm. In: Proc. of the 21st International Conference on Pattern Recognition (ICPR 2012), pp. 1213–1216 (2012)
Hammer, B., Strickert, M., Villmann, T.: Relevance LVQ versus SVM. In: Rutkowski, L., Siekmann, J.H., Tadeusiewicz, R., Zadeh, L.A. (eds.) ICAISC 2004. LNCS (LNAI), vol. 3070, pp. 592–597. Springer, Heidelberg (2004)
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Networks 15(8-9), 1059–1068 (2002)
Horn, R., Johnson, C.: Matrix Analysis, 2nd edn. Cambridge University Press (2013)
Kaden, M., Lange, M., Nebel, D., Riedel, M., Geweniger, T., Villmann, T.: Aspects in classification learning - Review of recent developments in Learning Vector Quantization. Foundations of Computing and Decision Sciences 39(2), 79–105 (2014)
Kaden, M., Riedel, M., Hermann, W., Villmann, T.: Border-sensitive learning in generalized learning vector quantization: an alternative to support vector machines. Soft Computing, page in press (2015)
Kohonen, T.: Learning vector quantization for pattern recognition. Report TKK-F-A601, Helsinki University of Technology, Espoo, Finland (1986)
Kohonen, T.: Learning Vector Quantization. Neural Networks 1(suppl. 1), 303 (1988)
Kohonen, T.: Self-Organizing Maps. Springer Series in Information Sciences, vol. 30. Springer, Heidelberg (1995) (2nd Extended Edition 1997)
Lange, M., Zühlke, D., Holz, O., Villmann, T.: Applications of l p -norms and their smooth approximations for gradient based learning vector quantization. In: Verleysen, M. (ed.) Proc. of European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2014), Louvain-La-Neuve, Belgium, pp. 271–276 (2014), i6doc.com
Leung, A., Chau, F., Gao, J.: A review on applications of wavelet transform techniques in chemical analysis: 1989–1997. Chemometrics and Intelligent Laboratory Systems 43(1), 165–184 (1998)
Liu, S., Trenkler, G.: Hadamard, Khatri-Rao, Kronecker and other matrix products. International Journal of Information and System Sciences 4(1), 160–177 (2008)
Osowski, S., Nghia, D.D.: Neural networks for classification of 2-d patterns. In: 2000 5th International Conference on Signal Processing Proceedings. 16th World Computer Congress 2000, WCC 2000—ICSP 2000, vol. 3, pp. 1568–1571. IEEE, Piscataway (2000)
Pekalska, E., Duin, R.: The Dissimilarity Representation for Pattern Recognition: Foundations and Applications. World Scientific (2006)
Sato, A., Yamada, K.: Generalized learning vector quantization. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Proceedings of the 1995 Conference on Advances in Neural Information Processing Systems 8, pp. 423–429. MIT Press, Cambridge (1996)
Schatten, R.: A Theory of Cross-Spaces. Annals of Mathematics Studies, vol. 26. Princeton University Press (1950)
Schölkopf, B., Smola, A.: Learning with Kernels. MIT Press, Cambridge (2002)
Schneider, P., Hammer, B., Biehl, M.: Adaptive relevance matrices in learning vector quantization. Neural Computation 21, 3532–3561 (2009)
Sonka, M., Hlavac, V., Boyle, R.: Image Processing, Analysis and Machine Vision, 2nd edn. Brooks Publishing (1998)
Strickert, M., Seiffert, U., Sreenivasulu, N., Weschke, W., Villmann, T., Hammer, B.: Generalized relevance LVQ (GRLVQ) with correlation measures for gene expression analysis. Neurocomputing 69(6-7), 651–659 (2006) ISSN: 0925-2312
Villmann, T., Haase, S.: Divergence based vector quantization. Neural Computation 23(5), 1343–1392 (2011)
Villmann, T., Haase, S., Kaden, M.: Kernelized vector quantization in gradient-descent learning. Neurocomputing 147, 83–95 (2015)
Walter, J., Arnrich, B., Scheering, C.: Learning fine positioning of a robot manipulator based on gabor wavelets. In: Proceedings of the International Joint Conference on Neural Networks, vol. 5, pp. 137–142. Univ. of Bielefeld, IEEE, Piscataway, NJ (2000)
Yoshimura, H., Etoh, M., Kondo, K., Yokoya, N.: Gray-scale character recognition by gabor jets projection. In: Proceedings of the 15th International Conference on Pattern Recognition, ICPR 2000, vol. 2, pp. 335–338. IEEE Comput. Soc., Los Alamitos (2000)
Zühlke, D., Schleif, F.-M., Geweniger, T., Haase, S., Villmann, T.: Learning vector quantization for heterogeneous structured data. In: Verleysen, M. (ed.) Proc. of European Symposium on Artificial Neural Networks (ESANN 2010), Evere, Belgium, pp. 271–276. d-side publications (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Bohnsack, A., Domaschke, K., Kaden, M., Lange, M., Villmann, T. (2015). Mathematical Characterization of Sophisticated Variants for Relevance Learning in Learning Matrix Quantization Based on Schatten-p-norms. In: Rutkowski, L., Korytkowski, M., Scherer, R., Tadeusiewicz, R., Zadeh, L., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2015. Lecture Notes in Computer Science(), vol 9119. Springer, Cham. https://doi.org/10.1007/978-3-319-19324-3_37
Download citation
DOI: https://doi.org/10.1007/978-3-319-19324-3_37
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19323-6
Online ISBN: 978-3-319-19324-3
eBook Packages: Computer ScienceComputer Science (R0)