Abstract
Word2Vec has become one of the most relevant neural networks to generate word embeddings for NLP applications. Despite that, little has been investigated in terms of its sensitivity to the word vectors’ length (n) and the window size (w). Thus, the present paper performs a sensitivity analysis of Word2Vec when applied to generate word embeddings for a deep neural architecture used to classify emotion states in tweets. Furthermore, we present a computational performance analysis to investigate how the system scales as a function of n and w in different computing environments. The results show that a window size of approximately half the tweet length (8 words) and a value of \(n = 50\) suffices to find good performances. Also, by increasing these values one may unnecessarily increase the computational cost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
De Castro, L.N., Ferrari, D.G.: Introduction to Data Mining: Basic Concepts, Algorithms, and Applications. Saraiva (2016). (in Portuguese)
Hinton, G.E.: Learning distributed representations of concepts. In: Proceedings of the Eight Annual Conference of the Cognitive Science Society (1986)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning representations by backpropagation errors. Nature 323, 533–536 (1986)
Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., Qin, B.: Learning sentiment-specific word embedding for Twitter sentiment classification. In: 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Maryland, USA (2014)
Zhou, G., He, T., Zhao, J., Hu, P.: Learning continuous word embedding with metadata for question retrieval in community question answering. In: 53nd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, China (2015)
Wang, P., Xu, B., Xu, J., Tian, G., Liu, C.-L., Hao, H.: Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification. Neurocomputing 174, 806–814 (2016)
Dos Santos, C., Gatti, M.: Deep convolutional neural networks for sentiment analysis of short texts. In: International Conference on Computational Linguistics (2014)
Koper, M., Kim, E., Klinger, R.: Emotion intensity prediction with affective norms, automatically extended resources and deep learning. In: Proceedings of the 8th Workshop on Computational Approaches in Subjectivy Sentiment and Social Media Analysis, pp. 50–57 (2017)
Goldberg, Y.: A primer on neural network models for natural language processing. J. Artif. Intell. Res. 57(2016), 345–420 (2016)
Sharma, R., Kaushik, P.: Literature survey of statistical, deep and reinforcement learning in natural language processing. In: 2017 International Conference on Computing, Communication and Automation (ICCCA) (2017)
Mulder, W.D., Bethard, S., Moens, M.-F.: A survey on the application of recurrent neural networks to statistical language modeling. Comput. Speech Lang. 30(1), 61–98 (2015)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: Proceedings of Workshop at ICLR (2013)
Liu, B.: Sentiment analysis and opinion mining. Synth. Lect. Hum. Lang. Technol. 5, 1–167 (2012). Morgan & Claypool Publishers
Pang, B., Lee, L.: Opinion mining and sentiment analysis. Found. Trends\(\textregistered \) Inf. Retrieval 2, 1–135 (2008)
Bucar, J., Povh, J.: Sentiment analysis in web text: an overview. In: Recent Advances in Information Science, pp. 154–159 (2013)
Thelwall, M., Buckley, K., Paltoglou, G., Cai, D.: Sentiment strength detection for the social Web. J. Am. Soc. Inf. Sci. Technol., 2544–2558 (2010)
Lima, A.C.E.S., de Castro, L.N.: Automatic sentiment analysis of Twitter messages. In: Proceedings of the Fourth International Conference on Computational Aspects of Social Networks (2012)
Weiyuan, L., Hua, X.: Text-based emotion classification using emotion cause extraction. Expert Syst. Appl. 41(4), 1742–1749 (2014)
Ekman, P., Friesen, W.V.: Constants across cultures in the face and emotion. J. Pers. Soc. Psychol. 17, 124–129 (1971)
Ekman, P., Friesen, W.V., Ellsworth, P.: Emotion in the Human Face, 1 edn., vol. 1 (1972). (A. P. Goldstein and L. Krasner, Eds., Pergamon)
Bellegarda, J.R.: Emotion analysis using latent affective folding and embedding. In: Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text, Los Angeles, California (2010)
Chaffar, S., Inkpen, D.: Using a heterogeneous dataset for emotion analysis in text. In: Proceedings of the 24th Canadian Conference on Advances in Artificial Intelligence, St. John’s, Canada (2011)
Dosciatti, M.M., Paterno, L., Paraiso, E.C.: Identificando Emoções em Textos em Português do Brasil usando Máquina de Vetores de Suporte em Solução Multiclasse,” Encontro Nacional de Inteligência Artificial e Computacional (ENIAC) (2013)
Wang, W., Chen, L., Thirunarayan, K., Sheth, A.P.: Harnessing Twitter ‘Big Data’ for automatic emoticon identification. In: 2012 International Conference on Social Computing, 11 January (2013)
Jurafsky, D., Martin, J.H.: Speech and Language Processing, 2nd edn. Prentice Hall, Upper Saddle River (2009)
Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (2014)
Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: AAAI 2015 Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Netw. 61, 85–117 (2015)
Da Silva, I.R.R., Lima, A.C.E.S., Pasti, R., De Castro, L.N.: Classifying emotions in twitter messages using a deep neural network. Springer (2018)
Da Silva, I.R.R., De Castro, L.N.: Estudos sobre um modelo de representação distribuída de palavras no contexto de análise de estados emocionais. Master Thesis (Master in Electrical and Computer Engineering) – Mackenzie Presbiteryan University, Sao Paulo (2018)
Acknowledgements
The authors thank CAPES, CNPq, Fapesp, and Mackpesquisa for the financial support. The authors also acknowledge the support of Intel for the Natural Computing and Machine Learning Laboratory as an Intel Center of Excellence in Artificial Intelligence.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pasti, R., Vilasbôas, F.G., Roque, I.R., de Castro, L.N. (2020). A Sensitivity and Performance Analysis of Word2Vec Applied to Emotion State Classification Using a Deep Neural Architecture. In: Herrera, F., Matsui , K., Rodríguez-González, S. (eds) Distributed Computing and Artificial Intelligence, 16th International Conference. DCAI 2019. Advances in Intelligent Systems and Computing, vol 1003 . Springer, Cham. https://doi.org/10.1007/978-3-030-23887-2_23
Download citation
DOI: https://doi.org/10.1007/978-3-030-23887-2_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-23886-5
Online ISBN: 978-3-030-23887-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)