Abstract
We present a method, which allows to train a Generalized Matrix Learning Vector Quantization (GMLVQ) model for classification using data from several, maybe non-calibrated, sources without explicit transfer learning. This is achieved by using a siamese-like GMLVQ-architecture, which comprises different sets of prototypes for the target classification and for the separation learning of the sources. In this architecture, a linear map is trained by means of GMLVQ for source distinction in the mapping space in parallel to the classification task learning. The respective null-space projection provides a common data representation of the different source data for an all-together classification learning.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Usually, sub-orthogonal matrices are defined in terms of column zero vectors or equivalently \(\boldsymbol{\varOmega }\in \mathbb {R}^{n\times m}\). Then, the more common relation \(\boldsymbol{\varOmega }^{T}\boldsymbol{\varOmega }=\mathbf {I}_{m}\) is equivalently valid.
- 2.
For the measurements a MCC-IMS-device from STEP Sensortechnik und Elektronik, Pockau, Germany (STEP IMS NOO) was used.
References
Biehl, M., Hammer, B., Villmann, T.: Prototype-based models in machine learning. Wiley Interdisc. Rev.: Cogn. Sci. 7(2), 92–111 (2016)
Bousmalis, K., Trigeorgis, G., Silberman, N., Krishnan, D., Erhan, D.: Domain separation networks. In: Proceedings of the 30th International Conference on Neural Information Processing Systems, pp. 343–351 (2016)
Bunte, K., Schneider, P., Hammer, B., Schleif, F.-M., Villmann, T., Biehl, M.: Limited rank matrix learning, discriminative dimension reduction and visualization. Neural Netw. 26(1), 159–173 (2012)
Crammer, K., Gilad-Bachrach, R., Navot, A., Tishby, A.: Margin analysis of the LVQ algorithm. In: Becker, S., Thrun, S., Obermayer, K. (eds.) Proceedings of NIPS 2002. Advances in Neural Information Processing, vol. 15, pp. 462–469. MIT Press, Cambridge (2003)
Crammer, K., Kearns, M., Wortman, J.: Learning from multiple sources. J. Mach. Learn. Res. 9(57), 1757–1774 (2008)
Ding, Z., Shao, M., Fu, Y.: Transfer learning for image classification with incomplete multiple sources. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), pp. 2188–2195. IEEE Press (2016)
Gama, J., Žliobaitė, I., Pechenizkiy, M., Bouchachia, A.: A survey on concept drift adaptation. ACM Comput. Surv. 46(4), 1–37 (2014)
Golub, G., Loan, C.V.: Matrix Computations. Johns Hopkins Studies in the Mathematical Sciences, 4th edn. John Hopkins University Press, Baltimore (2013)
Hastie, T., Simard, P., Säckinger, E.: Learning prototype models for tangent distance. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 999–1006. MIT Press (1995)
Heusinger, M., Raab, C., Schleif, F.-M.: Passive concept drift handling via variations of learning vector quantization. Neural Comput. Appl. 253 (2020)
Horn, R., Johnson, C.: Matrix Analysis, 2nd edn. Cambridge University Press, Cambridge (2013)
Kohonen, T.: Learning vector quantization. Neural Netw. 1(Supplement 1), 303 (1988)
Li, J., Wu, W., Xue, D., Gao, P.: Multi-source deep transfer neural network algorithm. Sensors 19(18), 3992–4008 (2019)
Lisboa, P., Saralajew, S., Vellido, A., Villmann, T.: The coming of age of interpretable and explainable machine learning models. In: Verleysen, M. (ed.) Proceedings of the 29th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2021), Bruges (Belgium), pp. 547–556, Louvain-La-Neuve, Belgium (2021). i6doc.com
Lu, J., Liu, A., Dong, F., Gu, F., Gama, J., Zhang, G.: Learning under concept drift: a review. IEEE Trans. Knowl. Data Eng. 31(12), 2346–2363 (2019)
Mathiasen, A., et al.: What if neural networks had SVDs? In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M.F., Lin, H. (eds.) Advances in Neural Information Processing Systems, vol. 33, pp. 18411–18420. Curran Associates Inc. (2020)
Paassen, B., Schulz, A., Hammer, B.: Linear supervised transfer learning for generalized matrix LVQ. Mach. Learn. Rep. 10(MLR-04-2016), 11–19 (2016)
Pan, S., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22(10), 1345–1359 (2010)
Prahm, C., Paassen, B., Schulz, A., Hammer, B., Aszmann, O.: Transfer learning for rapid re-calibration of a myoelectric prosthesis after electrode shift. In: Ibáñez, J., González-Vargas, J., Azorín, J.M., Akay, M., Pons, J.L. (eds.) Converging Clinical and Engineering Research on Neurorehabilitation II. BB, vol. 15, pp. 153–157. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-46669-9_28
Purkhart, R., Hillmann, A., Graupner, R., Becher, G.: Detection of characteristic clusters in IMS-spectrograms of exhaled air polluted with environmental contaminants. Int. J. Ion Mob. Spectromet. 15(15), 63–68 (2012)
Raab, C., Schleif, F.-M.: Transfer learning extensions for the probabilistic classification vector machine. Neurocomputing 397, 320–330 (2020)
Rudin, C.: Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead. Nat. Mach. Intell. 1(5), 206–215 (2019)
Rudin, C., Chen, C., Chen, Z., Huang, H., Semenova, L., Zhong, C.: Interpretable machine learning: fundamental principles and 10 grand challenges (2021)
Saralajew, S., Holdijk, L., Villmann, T.: Fast adversarial robustness certification of nearest prototype classifiers for arbitrary seminorms. In: Larochelle, H., Ranzato, M., Hadsell, R., Balcan, M., Lin, H. (eds.) Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020), vol. 33, pp. 13635–13650. Curran Associates Inc. (2020)
Saralajew, S., Nebel, D., Villmann, T.: Adaptive Hausdorff distances and tangent distance adaptation for transformation invariant classification learning. In: Hirose, A., Ozawa, S., Doya, K., Ikeda, K., Lee, M., Liu, D. (eds.) ICONIP 2016. LNCS, vol. 9949, pp. 362–371. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46675-0_40
Saralajew, S., Villmann, T.: Transfer learning in classification based on manifold models and its relation to tangent metric learning. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN), Anchorage, pp. 1756–1765. IEEE Computer Society Press (2017)
Sato, A., Yamada, K.: Generalized learning vector quantization. In: Touretzky, D.S., Mozer, M.C., Hasselmo, M.E. (eds.) Advances in Neural Information Processing Systems 8. Proceedings of the 1995 Conference, pp. 423–9. MIT Press, Cambridge (1996)
Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Trans. Neural Netw. 21(5), 831–840 (2010)
Schneider, P., Hammer, B., Biehl, M.: Adaptive relevance matrices in learning vector quantization. Neural Comput. 21, 3532–3561 (2009)
Simard, P., LeCun, Y., Denker, J.: Efficient pattern recognition using a new transformation distance. In: Hanson, S., Cowan, J., Giles, C. (eds.) Advances in Neural Information Processing Systems, vol. 5, pp. 50–58. Morgan-Kaufmann (1993)
Steppert, C., Steppert, I., Bollinger, T., Sterlacci, W.: Rapid non-invasive detection of influenza-A-infection by multicapillary column coupled ion mobility spectrometry. J. Breath Res. 15(1), 1–5 (2021)
Straat, M., Abadi, F., Göpfert, C., Hammer, B., Biehl, M.: Statistical mechanics of on-line learning under concept drift. Entropy 20(775), 1–21 (2018)
Raab, C., Schleif, F.-M.: Transfer learning extensions for the probabilistic classification vector machine. Neurocomputing 397, 320–330 (2020)
Sun, S., Shi, H., Wu, Y.: A survey of multi-source domain adaptation. Inf. Fusion 24, 84–92 (2015)
Tsai, J.-C., Chien, J.-T.: Adversarial domain separation and adaptation. In: Procceedings of the IEEE 27th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6 (2017)
Villmann, T., Saralajew, S., Villmann, A., Kaden, M.: Learning vector quantization methods for interpretable classification learning and multilayer networks. In: Sabourin, C., Merelo, J., Barranco, A., Madani, K., Warwick, K. (eds.) Proceedings of the 10th International Joint Conference on Computational Intelligence (IJCCI), Sevilla, pp. 15–21. SCITEPRESS - Science and Technology Publications, Lda, Lisbon (2018). ISBN 978-989-758-327-8
Yan, K., Zhang, D.: Correcting instrumental variation and time-varying drift: a transfer learning approach with autoencoders. IEEE Trans. Instrum. Meas. 65(9), 2012–2022 (2016)
Yang, Q., Zhang, Y., Dai, W., Pan, J.: Transfer Learning. Cambridge University Press, Cambridge (2020)
Zhuang, F., et al.: A comprehensive survey on transfer learning. Proc. IEEE 109(1), 43–76 (2021)
Acknowledgement
We thank the colleagues from Digitronic Chemnitz GmbH (Germany) for measuring the data. Further, D.S., M.K., and J.R. acknowledge funding by the European Social Fund (ESF) within a junior researcher group Maschinelles Lernen und Künstliche Intelligenz in Theorie und Anwnedung (MaLeKITA) and a PhD grant.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Villmann, T., Staps, D., Ravichandran, J., Saralajew, S., Biehl, M., Kaden, M. (2022). A Learning Vector Quantization Architecture for Transfer Learning Based Classification in Case of Multiple Sources by Means of Null-Space Evaluation. In: Bouadi, T., Fromont, E., Hüllermeier, E. (eds) Advances in Intelligent Data Analysis XX. IDA 2022. Lecture Notes in Computer Science, vol 13205. Springer, Cham. https://doi.org/10.1007/978-3-031-01333-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-01333-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-01332-4
Online ISBN: 978-3-031-01333-1
eBook Packages: Computer ScienceComputer Science (R0)