Skip to main content
SpringerLink
Log in

Word embedding for mixed-emotions analysis

  • Published:
Journal of Intelligent Information Systems Aims and scope Submit manuscript

Abstract

Word embedding is the process of converting words into vectors of real numbers which is of great interest in natural language processing. Recently, the performance of word embedding models has been the subject of some studies in emotion analysis. They mainly try to embed affective aspects of words into their vector representations utilizing some external sentiment/emotion lexica. The underlying emotion models in the existing studies follow basic emotion theories in psychology such as Plutchik or VAD. However, none of them investigate the Mixed Emotions (ME) model in their work which is the most precise theory of emotions raised in the recent psychological studies. According to ME, feelings can be the consequent of multiple emotion categories at the same time with different intensities. Relying on the ME model, this article embeds mixed emotions features into the existing word-vectors and performs extensive experiments on various English datasets. The analyses in both lines of intrinsic evaluations and extrinsic evaluations prove the improvement of the presented model over the existing emotion-aware embeddings such as SAWE and EWE.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://nlp.stanford.edu/projects/glove/

  2. https://code.google.com/p/word2vec

  3. https://www.dropbox.com/s/5egqnbktbfxp2im

  4. https://github.com/nbehzad/SAWE

References

  • Agrawal, A., An, A., & Papagelis, M. (2018). Learning emotion-enriched word representations. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 950–961).

  • Araque, O., Zhu, G., & Iglesias, C.A. (2019). A semantic similarity-based perspective of affect lexicons for sentiment analysis. Knowledge-Based Systems, 165, 346–359.

    Article  Google Scholar 

  • Berka, P. (2020). Sentiment analysis using rule-based and case-based reasoning. Journal of Intelligent Information Systems, 55, 51–66.

    Article  Google Scholar 

  • Berrios, R., Totterdell, P., & Kellett, S. (2015). Eliciting mixed emotions: a meta-analysis comparing models, types, and measures. Frontiers in Psychology, 6, 428.

    Article  Google Scholar 

  • Berrios, R., Totterdell, P., & Kellett, S. (2018). When feeling mixed can be meaningful: the relation between mixed emotions and eudaimonic well-being. Journal of Happiness Studies, 19, 841–861.

    Article  Google Scholar 

  • Bojanowski, P., Grave, E., Joulin, A., & Mikolov, T. (2017). Enriching word vectors with subword information. Transactions of the Association for Computational Linguistics, 5, 135–146.

    Article  Google Scholar 

  • Buechel, S., & Hahn, U. (2016). Emotion analysis as a regression problem—dimensional models and their implications on emotion representation and metrical evaluation. In Proceedings of the Twenty-second European Conference on Artificial Intelligence (pp. 1114–1122).

  • Chawla, K., Khosla, S., Chhaya, N., & Jaidka, K. (2019). Pre-trained affective word representations. In 2019 8th International Conference on Affective Computing and Intelligent Interaction (ACII) (pp. 1–7).

  • Chiu, B., Baker, S., Palmer, M., & Korhonen, A. (2019). Enhancing biomedical word embeddings by retrofitting to verb clusters. In Proceedings of the 18th BioNLP Workshop and Shared Task (pp. 125–134).

  • Devlin, J., Chang, M.-W., Lee, K., & Toutanova, K. (2019). BERT: pre-training of deep bidirectional transformers for language understanding. In NAACL-HLT (pp. 4171–4186).

  • Dobrakowski, A.G., Mykowiecka, A., Marciniak, M., Jaworski, W., & Biecek, P. (2021). Interpretable segmentation of medical free-text records based on word embeddings. Journal of Intelligent Information Systems, pp. 1–19.

  • Ekman, P. (1992). An argument for basic emotions. Cognition & Emotion, 6, 169–200.

    Article  Google Scholar 

  • Faruqui, M., Dodge, J., Jauhar, S.K., Dyer, C., Hovy, E., & Smith, N.A. (2015). Retrofitting word vectors to semantic lexicons. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (pp. 1606–1615).

  • Giatsoglou, M., Vozalis, M.G., Diamantaras, K., Vakali, A., Sarigiannidis, G., & Chatzisavvas, K.C. (2017). Sentiment analysis leveraging emotions and word embeddings. Expert Systems with Applications, 69, 214–224.

    Article  Google Scholar 

  • Gong, H., Bhat, S., Wu, L., Xiong, J., & W-m, Hwu (2019). Reinforcement learning based text style transfer without parallel training corpus. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 1(Long and Short Papers), 3168–3180.

    Google Scholar 

  • Izard, C.E. (2009). Emotion theory and research: Highlights, unanswered questions, and emerging issues. Annual Review of Psychology, 60, 1–25.

    Article  Google Scholar 

  • Khosla, S., Chhaya, N., & Chawla, K. (2018). Aff2vec: affect–enriched distributional word representations. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 2204–2218).

  • Labutov, I., & Lipson, H. (2013). Re-embedding words. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2:, Short Papers) (pp. 489–493).

  • Larsen, J.T., Coles, N.A., & Jordan, D.K. (2017). Varieties of mixed emotional experience. Current Opinion in Behavioral Sciences, 15, 72–76.

    Article  Google Scholar 

  • Maas, A., Daly, R.E., Pham, P.T., Huang, D., Ng, A.Y., & Potts, C. (2011). Learning word vectors for sentiment analysis (pp. 142–150).

  • Mehrabian, A. (1996). Pleasure-arousal-dominance: a general framework for describing and measuring individual differences in temperament. Current Psychology, 14, 261–292.

    Article  Google Scholar 

  • Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient estimation of word representations in vector space, presented at the ICLR Workshop.

  • Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., & Dean, J. (2013). Distributed representations of words and phrases and their compositionality. In Advances in Neural Information Processing Systems (pp. 3111–3119).

  • Mohamadi-Baghmolaei, R., Mozafari, N., & Hamzeh, A. (2015). Trust based latency aware influence maximization in social networks. Engineering Applications of Artificial Intelligence, 41, 195–206.

    Article  Google Scholar 

  • MohamadiBaghmolaei, R., Mozafari, N., & Hamzeh, A. (2017). Continuous states latency aware influence maximization in social networks. AI Communications, 30, 99–116.

    Article  MathSciNet  Google Scholar 

  • Mohammad, S. (2012). # Emotional tweets. In * SEM 2012: The First Joint Conference on Lexical and Computational Semantics–Volume 1: Proceedings of the Main Conference and the Shared Task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012) (pp. 246–255).

  • Mohammad, S. (2018). Word affect intensities. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC) (p. 2018).

  • Mohammad, S., Bravo-Marquez, F., Salameh, M., & Kiritchenko, S. (2018). Semeval-2018 task 1: affect in tweets. In Proceedings of the 12th International Workshop on Semantic Evaluation (pp. 1–17).

  • Mohammad, S.M., & Kiritchenko, S. (2015). Using hashtags to capture fine emotion categories from tweets. Computational Intelligence, 31, 301–326.

    Article  MathSciNet  Google Scholar 

  • Mohammad, S., & Turney, P. (2010). Emotions evoked by common words and phrases: Using mechanical turk to create an emotion lexicon. In Proceedings of the NAACL HLT 2010 Workshop on Computational Approaches to Analysis and Generation of Emotion in Text (pp. 26–34).

  • Mohammad, S.M., & Turney, P.D. (2013). Crowdsourcing a word–emotion association lexicon. Computational Intelligence, 29, 436–465.

    Article  MathSciNet  Google Scholar 

  • MohammadiBaghmolaei, R., & Ahmadi, A. (2020). Word embedding for emotional analysis: an overview. In 2020 28th Iranian Conference on Electrical Engineering (ICEE) (pp. 1–5).

  • Mrksic, N., Seaghdha, D., Thomson, B., Gasic, M., Rojas-Barahona, L., Su, P., & et al. (2016). Counter-fitting word vectors to linguistic constraints. In 2016 Conference of the North American Chapter of the Association for Computational linguistics: Human Language Technologies, NAACL HLT 2016-Proceedings of the Conference (pp. 142–148).

  • Naderalvojoud, B., & Sezer, E.A. (2020). Sentiment aware word embeddings using refinement and senti-contextualized learning approach. Neurocomputing, 405, 149–160.

    Article  Google Scholar 

  • Oramas Bustillos, R., Zatarain Cabada, R., Barrón Estrada, M.L., & Hernández Pérez, Y. (2019). Opinion mining and emotion recognition in an intelligent learning environment. Computer Applications in Engineering Education, 27, 90–101.

    Article  Google Scholar 

  • Parker, R., Graff, D., Kong, J., Chen, K., & Maeda, K. (2011). English gigaword fifth edition LDC2011T07. Web Download. Philadelphia: Linguistic Data Consortium.

  • Pennington, J., Socher, R., & Manning, C.D. (2014). Glove: global vectors for word representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 1532–1543).

  • Peters, M.E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., & et al. (2018). Deep contextualized word representations. In Proceedings of NAACL-HLT (pp. 2227–2237).

  • Plutchik, R. (1994). The psychology and biology of emotion. HarperCollins College Publishers.

  • Rezaeinia, S.M., Rahmani, R., Ghodsi, A., & Veisi, H. (2019). Sentiment analysis based on improved pre-trained word embeddings. Expert Systems with Applications, 117, 139–147.

    Article  Google Scholar 

  • Serban, I.V., Sordoni, A., Lowe, R., Charlin, L., Pineau, J., Courville, A.C., & et al. (2017). A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 31.

  • Seyeditabari, A., Tabari, N., Gholizadeh, S., & Zadrozny, W. (2019). Emotional embeddings:, refining word embeddings to capture emotional content of words. arXiv:1906.00112.

  • Socher, R., Bauer, J., Manning, C.D., & Ng, A.Y. (2013). Parsing with compositional vector grammars. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1:, Long Papers) (pp. 455–465).

  • Socher, R., Pennington, J., Huang, E.H., Ng, A.Y., & Manning, C.D. (2011). Semi-supervised recursive autoencoders for predicting sentiment distributions. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (pp. 151–161).

  • Staiano, J., & Guerini, M. (2014). Depeche mood: a lexicon for emotion analysis from crowd annotated news. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2:, Short Papers) (pp. 427–433).

  • Tang, D., Wei, F., Qin, B., Yang, N., Liu, T., & Zhou, M. (2015). Sentiment embeddings with applications to sentiment analysis. IEEE Transactions on Knowledge and Data Engineering, 28, 496–509.

    Article  Google Scholar 

  • Tang, D., Wei, F., Yang, N., Zhou, M., Liu, T., & Qin, B. (2014). Learning sentiment-specific word embedding for twitter sentiment classification (pp. 1555–1565).

  • Tarnowska, K.A., & Ras, Z.W. (2019). Sentiment analysis of customer data. In Web Intelligence (pp. 343–363).

  • Tarnowska, K.A., & Ras, Z. (2021). NLP-Based customer loyalty improvement recommender system (CLIRS2). Big Data and Cognitive Computing, 5, 4.

    Article  Google Scholar 

  • Teofili, T., & Chhaya, N. (2019). Affect enriched word embeddings for news information retrieval. arXiv:1909.01772.

  • Wallbott, H.G., & Scherer, K.R. (1986). How universal and specific is emotional experience? Evidence from 27 countries on five continents. Social Science Information, 25, 763–795.

    Article  Google Scholar 

  • Wang, S., Maoliniyazi, A., Wu, X., & Meng, X. (2020). Emo2vec: Learning emotional embeddings via multi-emotion category. ACM Transactions on Internet Technology (TOIT), 20, 1–17.

    Google Scholar 

  • Warriner, A.B., Kuperman, V., & Brysbaert, M. (2013). Norms of valence, arousal, and dominance for 13,915 English lemmas. Behavior Research Methods, 45, 1191–1207.

    Article  Google Scholar 

  • Wikipedia dumps. Available: https://dumps.wikimedia.org/. Accessed 2014.

  • Wilson, T., Wiebe, J., & Hoffmann, P. (2005). Recognizing contextual polarity in phrase-level sentiment analysis. In Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (pp. 347–354).

  • Wu, Y., Wu, W., Xing, C., Xu, C., Li, Z., & Zhou, M. (2019). A sequential matching framework for multi-turn response selection in retrieval-based chatbots. Computational Linguistics, 45, 163–197.

    Article  MathSciNet  Google Scholar 

  • Wu, D., Yang, R., & Shen, C. (2021). Sentiment word co-occurrence and knowledge pair feature extraction based LDA short text clustering algorithm. Journal of Intelligent Information Systems, 56, 1–23.

    Article  Google Scholar 

  • Ye, Z., Li, F., & Baldwin, T. (2018). Encoding sentiment information into word vectors for sentiment analysis. In Proceedings of the 27th International Conference on Computational Linguistics (pp. 997–1007).

  • Yilmaz, S., & Toklu, S. (2020). A deep learning analysis on question classification task using word2vec representations. Neural Computing and Applications, pp. 1–20.

  • Zhao, X., Zhang, Y., Guo, W., & Yuan, X. (2018). Jointly trained convolutional neural networks for online news emotion analysis. In International Conference on Web Information Systems and Applications (pp. 170–181).

Download references

Author information

Authors and Affiliations

  1. Faculty of Computer Engineering, K.N. Toosi University of Technology, Tehran, Iran

    Rezvan MohammadiBaghmolaei & Ali Ahmadi

Authors
  1. Rezvan MohammadiBaghmolaei
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Ahmadi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rezvan MohammadiBaghmolaei.

Ethics declarations

Conflict ofinterests

The authors declare that they have no conflict of interest.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

MohammadiBaghmolaei, R., Ahmadi, A. Word embedding for mixed-emotions analysis. J Intell Inf Syst 60, 49–72 (2023). https://doi.org/10.1007/s10844-022-00720-w

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10844-022-00720-w

Keywords

Search

Navigation