Abstract
The paper investigates the problem of recognizing human emotions by voice using deep learning methods. Deep convolutional neural networks and recurrent neural networks with bidirectional LSTM memory cell were used as models of deep neural networks. On their basis, an ensemble of neural networks is proposed. We carried out computer experiments on using the constructed neural networks and popular machine learning algorithms for recognizing emotions in human speech contained in the RAVDESS audio record database. The computational results showed a higher efficiency of neural network models compared to machine learning algorithms. Accuracy estimates for individual emotions obtained using neural networks were 80%. The directions of further research in the field of recognition of human emotions are proposed.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Deep learning library (2020). https://pytorch.org/. Accessed 4 Oct 2020
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. CoRR abs/1512.03385 (2015). http://arxiv.org/abs/1512.03385
Hochreiter, S., Bengio, Y., Frasconi, P., Schmidhuber, J.: Gradient flow in recurrent nets: the difficulty of learning long-term dependencies. In: Kremer, S.C., Kolen, J.F. (eds.) A Field Guide to Dynamical Recurrent Neural Networks. IEEE Press (2001)
Ishi, C., Ishiguro, H., Hagita, N.: Using prosodic and voice quality features for paralinguistic information extraction. In: Proceedings of the Speech Prosody 2006, pp. 883–886, Dresden (2006)
Karpov, A.A., Kaya, H., Salakh, A.A.: Actual problems and achievements of paralinguistic speech analysis. Nauchno-tekhnicheskiy vestnik informatsionnykh tekhnologiy, mekhaniki i optiki 16(4), 581–592 (2016). (in Russian)
Kennedy, L., Ellis, D.: Pitch-based emphasis detection for characterization of meeting recordings. In: Proceedings of the ASRU, pp. 243–248, Virgin Islands (2003)
Kockmann, M., Burget, L., Cernock, J.: Brno university of technology system for interspeech 2010 paralinguistic challenge, pp. 2822–2825 (2010)
Kurkov, N.A., Shchetinin, E.Y.: Emotion classification by voice using the blstm neural network. In: Information and Telecommunication Technologies and Mathematical Modeling of High-Tech Systems, pp. 461–464 (2019)
Lee, C., Narayanan, S., Pieraccini, R.: Recognition of negative emotions from the speech signal, pp. 240–243 (2001)
Liu, J., Chen, C., Bu, J., You, M., Tao, J.: Speech Emotion Recognition Using an Enhanced Co-training Algorithm, pp. 999–1002. Springer, Heidelberg (2007). https://doi.org/10.1109/ICME.2007.4284821
Livingstone, S.R., Russo, F.A.: The Ryerson audio-visual database of emotional speech and song (Ravdess): a dynamic, multimodal set of facial and vocal expressions in North American English. PLoS ONE 13(5), 1–35 (2018). https://doi.org/10.1371/journal.pone.0196391
Popova, A.S., Rassadin, A.G., Ponomarenko, A.A.: Emotion recognition in sound. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V. (eds.) NEUROINFORMATICS 2017. SCI, vol. 736, pp. 117–124. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-66604-4_18
Rabiner, L., Juang, B.: Fundamental of Speech Recognition. Prentice-Hall, Englewood Cliffs (1993)
Schuller, B.: The computational paralinguistics challenge. IEEE Signal Process. Mag. 29(4), 1264–1281 (2012)
Schuller, B., Batliner, A.: Computational Paralinguistics: Emotion Affect and Personality in Speech and Language Processing. Wiley, New York (2013)
Schuster, M., Paliwal, K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45, 2673–2681 (1997). https://doi.org/10.1109/78.650093
Sevastyanov, L.A., Shchetinin, E.Y.: On methods of increasing the accuracy of multiclass classification based on unbalanced data. Inf. Appl. 14(1), 67–74 (2020). https://doi.org/10.14357/19922264200109
Singh, N., Agrawal, A., Khan, R.A.: Automatic speaker recognition: current approaches and progress in last six decades. Global J. Enterp. Inf. Syst. 9(3), 45–52 (2017). https://doi.org/10.18311/gjeis/2017/15973
Steidl, S.: Automatic Classification of Emotion-Related User States in Spontaneous Children’s Speech. Logos Verlag, Berlin (2009)
Sterling, G., Prikhodko, P.: Deep learning in the problem of recognizing emotions from speech. In: Proceedings of the Conference Information Technologies and Systems 2016 IITP RAS. pp. 451–456 (2016). (in Russian)
Acknowledgments
The publication has been prepared with the support of the. “RUDN University Program 5-100” and funded by Russian Foundation for Basic Research (RFBR) according to the research project No 19-01-00645.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Shchetinin, E.Y., Sevastianov, L.A., Kulyabov, D.S., Ayrjan, E.A., Demidova, A.V. (2020). Deep Neural Networks for Emotion Recognition. In: Vishnevskiy, V.M., Samouylov, K.E., Kozyrev, D.V. (eds) Distributed Computer and Communication Networks. DCCN 2020. Lecture Notes in Computer Science(), vol 12563. Springer, Cham. https://doi.org/10.1007/978-3-030-66471-8_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-66471-8_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-66470-1
Online ISBN: 978-3-030-66471-8
eBook Packages: Computer ScienceComputer Science (R0)