Abstract
The furtherance of social media led people to share the reviews in various ways such as video, audio and text. Recently, the performance of sentiment classification is achieved success using neural networks. In this paper, neural network approach is presented to detect the sentiment from audio and text models. For audio, features like Mel Frequency Cepstral Coefficients (MFCC) are used to build Deep Neural Network (DNN) and Deep Neural Network Attention Mechanism (DNNAM) classifiers. From the results, it is noticed that DNNAM gives better results compared to DNN because the DNN is a frame based one where as the DNNAM is an utterance level classification thereby efficiently use the context. Additionally, textual features are extracted from the transcript of the audio input using Word2vec model. Support Vector Machine (SVM) and Long Short-Term Memory Recurrent Neural Network (LSTM-RNN) classifiers are used to develop a sentiment model. From the experiments it is noticed the LSTM-RNN outperforms the SVM as the LSTM-RNN is able to memorize long temporal context. The performance is also significantly improved by combining both the audio and text modalities.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kumar, A., Sebastian, T.M.: Sentiment analysis on Twitter. Int. J. Comput. Sci. (IJCSI) 9(4), 372–378 (2012)
Agarwal, A., Xie, B., Vovsha, I., Rambow, O., Passonneau, R.: Sentiment analysis of Twitter data. In: Proceedings of Workshop on Languages in Social Media, pp. 30–38 (2011)
Patra, B.G., Das, D., Bandyopadhyay, S.: Mood classification of Hindi songs based on lyrics. In: Proceedings of 12th International Conference on Natural Language Processing (ICON) (2015)
dos Santos, C.N., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: Proceedings of 25th International Conference on Computational Linguistics (COLING), pp. 69–78 (2014)
Raffel, C., Ellis, D.P.W.: Feed-forward networks with attention can solve some long-term memory problems. In: CoRR, vol. abs/1512.08756 (2015). http://arxiv.org/abs/1512.08756
Mairesse, F., Polifroni, J., Di Fabbrizio, G.: Can prosody inform sentiment analysis? Experiments on short spoken reviews. In: Proceedings of IEEE International Confernce on Acoustics, Speech, Signal processing (ICASSP), pp. 5093–5096 (2012)
Richardson, F., Reynolds, D., Dehak, N.: A unified deep neural network for speaker, language recognition. In: Proceedings of INTERSPEECH, pp. 1146–1150 (2015)
Hinton, G., Deng, L., Dong, Y., Dahl, G., Mohamed, A., Jaitly, N., Senior, A., Vanhoucke, V., Nguyen, P., Sainath, T., Kingsbury, B.: Deep neural networks for acoustic modeling in speech recognition. IEEE Signal Process. Mag. 29(6), 82–97 (2012)
Abburi, H., Akkireddy, E.S.A., Gangashetty, S.V., Mamidi, R.: Multimodal sentiment analysis of Telugu songs. In: Proceedings of 4th Workshop on Sentiment Analysis where AI meets Psychology (SAAIP), pp. 48–52 (2016)
Abburi, H., Prasath, R., Shrivastava, M., Gangashetty, S.V.: Multimodal sentiment analysis using deep neural networks. In: Prasath, R., Gelbukh, A. (eds.) MIKE 2016. LNCS (LNAI), vol. 10089, pp. 58–65. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-58130-9_6
Lopez-Moreno, I., Gonzalez-Dominguez, J., Plchot, O., Martinez, D., Gonzalez-Rodriguez, J., Moreno, P.: Automatic language identification using deep neural networks. In: Proceedings of IEEE International Conference on Acoustic, Speech, Signal Processing (ICASSP), pp. 5337–5341 (2014)
Deng, L.: A tutorial survey of architectures, algorithms, applications for deep learning. APSIPA Trans. Signal Inf. Process. 3, 1–29 (2014)
Morency, L.P., Mihalcea, R., Doshi, P.: Towards multimodal sentiment analysis: harvesting opinions from the web. In: Proceedings of 13th International Conference on Multimodal Interfaces (ICMI), pp. 169–176, November 2011
Mounika, K.V., Sivanand, A., Lakshmi, H.R., Gangashetty, S.V., Vuppala, A.K.: An investigation of deep neural network architectures for language recognition in Indian languages. In: Proceedings of INTERSPEECH, pp. 2930–2933 (2016)
Gamallo, P., Garcia, M.: Citius: a naive-bayes strategy for sentiment analysis on English Tweets. In: Proceedings of 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 171–175, August 2014
Singh, R., Kaur, R.: Sentiment analysis on social media, online review. Int. J. Comput. Appl. 121(20), 44–48 (2015)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Poria, S., Cambria, E., Howard, N., Huang, G.-B., Hussain, A.: Fusing audio, visual, textual clues for sentiment analysis from multimodal content. Neurocomputing 174, 50–59 (2015)
Mikolov, T., Karafiat, M., Burget, L., Cernocky, J.H., Khudanpur, S.: Recurrent neural network based language model. In: Proceedings of INTERSPEECH, pp. 1045–1048 (2010)
Perez-Rosas, V., Mihalcea, R., Morency, L.-P.: Multimodal sentiment analysis of Spanish online videos. IEEE Intell. Syst. 28(3), 38–45 (2013)
Perez-Rosas, V., Mihalcea, R., Morency, L.-P.: Utterance level multimodal sentiment analysis. In: Proceedings of ACL, pp. 973–982 (2013)
Medhat, W., Hassan, A., Korashy, H.: Sentiment analysis algorithms, applications: a survey. Ain Shams Eng. J. 5(4), 1093–1113 (2014)
Fang, X., Zhan, J.: Sentiment analysis using product review data. J. Big Data 2(5), 1–14 (2015). Springer Open Journal
Bengio, Y., Ducharme, R., Vincent, P., Jauvin, C.: A neural probabilistic language model. J. Mach. Learn. Res. 3, 1137–1155 (2003)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Abburi, H., Prasath, R., Shrivastava, M., Gangashetty, S.V. (2017). Significance of DNN-AM for Multimodal Sentiment Analysis. In: Ghosh, A., Pal, R., Prasath, R. (eds) Mining Intelligence and Knowledge Exploration. MIKE 2017. Lecture Notes in Computer Science(), vol 10682. Springer, Cham. https://doi.org/10.1007/978-3-319-71928-3_23
Download citation
DOI: https://doi.org/10.1007/978-3-319-71928-3_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-71927-6
Online ISBN: 978-3-319-71928-3
eBook Packages: Computer ScienceComputer Science (R0)