Skip to main content
SpringerLink
Log in

Direct Incorporation of \(L_1\)-Regularization into Generalized Matrix Learning Vector Quantization

  • Conference paper
  • First Online:
Book cover Artificial Intelligence and Soft Computing (ICAISC 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10841))

Included in the following conference series:

  • 2163 Accesses

Abstract

Frequently, high-dimensional features are used to represent data to be classified. This paper proposes a new approach to learn interpretable classification models from such high-dimensional data representation. To this end, we extend a popular prototype-based classification algorithm, the matrix learning vector quantization, to incorporate an enhanced feature selection objective via \(L_1\)-regularization. In contrast to previous work, we propose a framework that directly optimizes this objective using the alternating direction method of multipliers (ADMM) and manifold optimization. We evaluate our method on synthetic data and on real data for speech-based emotion recognition. Particularly, we show that our method achieves state-of-the-art results on the Berlin Database of Emotional speech and show its abilities to select relevant dimensions from the eGeMAPS set of audio features.

F. Lischke—This work was supported in part by SAB grant number 100231931.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2009)

    MATH  Google Scholar 

  2. Ali, H., Hariharan, M., Yaacob, S., Adom, A.H.: Facial emotion recognition using empirical mode decomposition. Expert Syst. Appl. 42(3), 1261–1277 (2015)

    Article  Google Scholar 

  3. Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)

    Article  MathSciNet  Google Scholar 

  4. Bi, J., Bennett, K., Embrechts, M., Breneman, C., Song, M.: Dimensionality reduction via sparse support vector machines. JMLR 3(Mar), 1229–1243 (2003)

    MATH  Google Scholar 

  5. Biehl, M., Hammer, B., Villmann, T.: Prototype-based models in machine learning. Wiley Interdisc. Rev.: Cogn. Sci. 7(2), 92–111 (2016)

    Article  Google Scholar 

  6. Biehl, M., Hammer, B., Schleif, F.M., Schneider, P., Villmann, T.: Stationarity of matrix relevance learning vector quantization. Mach. Learn. Rep. 3, 1–17 (2009)

    Google Scholar 

  7. Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, New York (2006)

    MATH  Google Scholar 

  8. Bojer, T., Hammer, B., Schunk, D., Von Toschanowitz, K.: Relevance determination in learning vector quantization. In: Proceedings of ESANN (2001)

    Google Scholar 

  9. Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)

    Article  Google Scholar 

  10. Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A database of German emotional speech. In: Interspeech, vol. 5, pp. 1517–1520 (2005)

    Google Scholar 

  11. Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)

    Article  Google Scholar 

  12. Donoho, D.L.: De-noising by soft-thresholding. IEEE TIT 41(3), 613–627 (1995)

    MathSciNet  MATH  Google Scholar 

  13. Donoho, D.L.: For most large underdetermined systems of linear equations the minimal \(\ell \)1-norm solution is also the sparsest solution. CPAMA 59(6), 797–829 (2006)

    MathSciNet  MATH  Google Scholar 

  14. Eyben, F., Scherer, K.R., Schuller, B.W., Sundberg, J., André, E., Busso, C., Devillers, L.Y., Epps, J., Laukka, P., Narayanan, S.S., Truong, K.P.: The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE TAC 7(2), 190–202 (2016)

    Google Scholar 

  15. Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in openSMILE, the munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM, pp. 835–838. ACM (2013)

    Google Scholar 

  16. Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8), 1059–1068 (2002)

    Article  Google Scholar 

  17. Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE TNN 13(2), 415–425 (2002)

    Google Scholar 

  18. Kaden, M., Lange, M., Nebel, D., Riedel, M., Geweniger, T., Villmann, T.: Aspects in classification learning - review of recent developments in learning vector quantization. Found. Comput. Decis. Sci. 39(2), 79–105 (2014)

    Article  MathSciNet  Google Scholar 

  19. Kanth, N.R., Saraswathi, S.: Efficient speech emotion recognition using binary support vector machines multiclass SVM. In: 2015 IEEE ICCIC, December 2015

    Google Scholar 

  20. Kim, J., Truong, K.P., Englebienne, G., Evers, V.: Learning spectro-temporal features with 3D CNNs for speech emotion recognition. arXiv preprint arXiv:1708.05071 (2017)

  21. Kohonen, T.: Learning vector quantization. In: Kohonen, T. (ed.) Self-Organizing Maps. SSINF, vol. 30, pp. 175–189. Springer, Heidelberg (1995). https://doi.org/10.1007/978-3-642-97610-0_6

    Chapter  Google Scholar 

  22. Korkmaz, O.E., Atasoy, A.: Emotion recognition from speech signal using mel-frequency cepstral coefficients. In: 2015 9th ELECO, pp. 1254–1257, November 2015

    Google Scholar 

  23. Lee, J., Tashev, I.: High-level feature representation using recurrent neural network for speech emotion recognition. In: Interspeech 2015. ISCA, September 2015

    Google Scholar 

  24. Mao, Q., Dong, M., Huang, Z., Zhan, Y.: Learning salient features for speech emotion recognition using convolutional neural networks. IEEE Trans. Multimedia 16(8), 2203–2213 (2014)

    Article  Google Scholar 

  25. Murty, K.G., Kabadi, S.N.: Some NP-complete problems in quadratic and nonlinear programming. Math. Program. 39(2), 117–129 (1987)

    Article  MathSciNet  Google Scholar 

  26. Ng, A.Y.: Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Proceedings of the 21th ICML, ICML 2004, p. 78. ACM, New York (2004)

    Google Scholar 

  27. Obozinski, G., Taskar, B., Jordan, M.: Multi-task feature selection. Statistics Department, UC Berkeley, Technical report 2 (2006)

    Google Scholar 

  28. Ofodile, I., Kulkarni, K., Corneanu, C.A., Escalera, S., Baro, X., Hyniewska, S., Allik, J., Anbarjafari, G.: Automatic recognition of deceptive facial expressions of emotion (2017)

    Google Scholar 

  29. Palo, H., Mohanty, M., Chandra, M.: Efficient feature combination techniques for emotional speech classification. IJST 19(1), 135–150 (2016)

    Google Scholar 

  30. Riedel, M., Rossi, F., Kästner, M., Villmann, T.: Regularization in relevance learning vector quantization using \(l_1\)-norms. In: Verleysen, M. (ed.) Proceedings of ESANN 2013, pp. 17–22 (2013)

    Google Scholar 

  31. Saeys, Y., Abeel, T., Van de Peer, Y.: Robust feature selection using ensemble feature selection techniques. In: Daelemans, W., Goethals, B., Morik, K. (eds.) ECML PKDD 2008. LNCS (LNAI), vol. 5212, pp. 313–325. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-87481-2_21

    Chapter  Google Scholar 

  32. Sato, A., Yamada, K.: Generalized learning vector quantization. In: Advances in Neural Information Processing Systems, pp. 423–429 (1996)

    Google Scholar 

  33. Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization. Neural Comput. 21(12), 3532–3561 (2009)

    Article  MathSciNet  Google Scholar 

  34. Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Trans. Neural Netw. 21(5), 831–840 (2010)

    Article  Google Scholar 

  35. Schuller, B., Steidl, S., Batliner, A.: The INTERSPEECH 2009 emotion challenge. In: 10th Annual Conference of the ISCA (2009)

    Google Scholar 

  36. Schuller, B., Steidl, S., Batliner, A., Schiel, F., Krajewski, J.: The INTERSPEECH 2011 speaker state challenge. In: 12th Annual Conference of the ISCA (2011)

    Google Scholar 

  37. Schuller, B., Steidl, S., Batliner, A., et al.: The INTERSPEECH 2017 computational paralinguistics challenge: addressee, cold and snoring. In: ComParE, Interspeech 2017, pp. 3442–3446 (2017)

    Google Scholar 

  38. Sinith, M.S., Aswathi, E., Deepa, T.M., Shameema, C.P., Rajan, S.: Emotion recognition from audio signals using support vector machine. In: 2015 IEEE RAICS, pp. 139–144, December 2015

    Google Scholar 

  39. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  40. Townsend, J., Koep, N., Weichwald, S.: Pymanopt: a python toolbox for optimization on manifolds using automatic differentiation. J. Mach. Learn. Res. 17(137), 1–5 (2016)

    MathSciNet  MATH  Google Scholar 

  41. Villmann, T., Bohnsack, A., Kaden, M.: Can learning vector quantization be an alternative to SVM and deep learning? JAISCR 7(1), 65–81 (2017)

    Google Scholar 

  42. Wang, K., An, N., Li, B.N., Zhang, Y., Li, L.: Speech emotion recognition using fourier parameters. IEEE TAC 6(1), 69–75 (2015)

    Google Scholar 

  43. Wen, G., Li, H., Huang, J., Li, D., Xun, E.: Random deep belief networks for recognizing emotions from speech signals. Comput. Intell. Neurosci. 2017 (2017)

    Article  Google Scholar 

  44. Zhang, Y., Zhang, L., Hossain, M.A.: Adaptive 3D facial action intensity estimation and emotion recognition. Expert Syst. Appl. 42(3), 1446–1464 (2015)

    Article  Google Scholar 

  45. Zhu, J., Rosset, S., Tibshirani, R., Hastie, T.J.: 1-norm support vector machines. In: Advances in Neural Information Processing Systems, pp. 49–56 (2004)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. HTW Dresden, Friedrich-List-Platz 1, 01069, Dresden, Germany

    Falko Lischke, Thomas Neumann, Sven Hellbach & Hans-Joachim Böhme

  2. Saxony Institute for Computational Intelligence and Machine Learning, University of Applied Sciences Mittweida, 09648, Mittweida, Germany

    Thomas Villmann

Authors
  1. Falko Lischke
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Thomas Neumann
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Sven Hellbach
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas Villmann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Hans-Joachim Böhme
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Falko Lischke .

Editor information

Editors and Affiliations

  1. Częstochowa University of Technology, Częstochowa, Poland

    Leszek Rutkowski

  2. Częstochowa University of Technology, Częstochowa, Poland

    Rafał Scherer

  3. Częstochowa University of Technology, Częstochowa, Poland

    Marcin Korytkowski

  4. University of Alberta, Edmonton, AB, Canada

    Witold Pedrycz

  5. AGH University of Science and Technology, Kraków, Poland

    Ryszard Tadeusiewicz

  6. University of Louisville, Louisville, KY, USA

    Jacek M. Zurada

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lischke, F., Neumann, T., Hellbach, S., Villmann, T., Böhme, HJ. (2018). Direct Incorporation of \(L_1\)-Regularization into Generalized Matrix Learning Vector Quantization. In: Rutkowski, L., Scherer, R., Korytkowski, M., Pedrycz, W., Tadeusiewicz, R., Zurada, J. (eds) Artificial Intelligence and Soft Computing. ICAISC 2018. Lecture Notes in Computer Science(), vol 10841. Springer, Cham. https://doi.org/10.1007/978-3-319-91253-0_61

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-91253-0_61

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-91252-3

  • Online ISBN: 978-3-319-91253-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation