research-article

Emo2Vec: Learning Emotional Embeddings via Multi-Emotion Category

Authors:

Aishan Maoliniyazi,

Xiaofeng MengAuthors Info & Claims

ACM Transactions on Internet Technology (TOIT), Volume 20, Issue 2

Article No.: 13, Pages 1 - 17

https://doi.org/10.1145/3372152

Published: 17 April 2020 Publication History

Abstract

Sentiment analysis or opinion mining for subject information extraction from the text has become more and more dependent on natural language processing, especially for business and healthcare, since the online products and service reviews affect the consuming behaviors. Word embeddings that can map the words to low-dimensional vector representations have been widely used in natural language processing tasks. But the word embeddings based on context such as Word2Vec and GloVe fail to capture the sentiment information. Most of existing sentiment analysis methods incorporate emotional polarity (positive and negative) to improve the sentiment embeddings for the emotion classification. This article takes advantage of an emotional psychology model to learn the emotional embeddings in Chinese first. In order to combine the semantic space and an emotional space, we present two different purifying models from local (LPM) and global (GPM) perspectives based on Plutchik's wheel of emotions to add the emotional information into word vectors. The two models aim to improve the word vectors so that not only the semantically similar words but also the sentimentally similar words can be closer than before. The Plutchik's wheel of emotions model can give eight-dimensional vector for one word in emotional space that can capture more sentiment information than the binary polarity labels. The obvious advantage of the local purifying model is that it can be fit for any pretrained word embeddings. For the global purifying model, we can get the final emotional embeddings at once. These models have been extended to handle English texts. The experimental results on Chinese and English datasets show that our purifying model can improve the conventional word embeddings and some proposed sentiment embeddings for sentiment classification and multi-emotion classification.

References

[1]

R. Plutchik. 2001. The nature of emotions. American Scientist, 89(4).

[2]

Y. Bengio, R. Ducharme, P. Vincent, and C. Jauvin. 2003. A neural probabilistic language model. Journal of Machine Learning Research 3 (2003), 1137--1155.

Digital Library

[3]

R. Collobert and J. Weston. 2008. A unified architecture for natural language processing: Deep neural networks with multi task learning. In Proc. Int. Conf. Machine Learning. 160--167.

[4]

A. Mnih and G. E. Hinton 2008. A scalable hierarchical distributed language model. Neural Information Processing Systems. 1081--1088.

[5]

T. Mikolov, K. Chen, G. Corrado, and J. Dean. 2013. Efficient estimation of word representations in vector space. In Proceedings of the International Conference on Learning Representations. 1--12.

[6]

O. Levy and Y. Goldberg. 2014. Dependency-based word embeddings. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Baltimore, Maryland, 302--308.

[7]

Y. Liu, Z. Liu, T.-S. Chua, and M. Sun. 2015. Topical word embeddings. Association for the Advancement of Artificial Intelligence. 2418--2424.

[8]

A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Ng, and C. Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Portland, Oregon, 142--150.

[9]

I. Labutov and H. Lipson. 2013. Re-embedding words. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), Sofia, Bulgaria 489--493.

[10]

D. Tang, F. Wei, B. Qin, N. Yang, T. Liu, and M. Zhou. 2016. Sentiment embeddings with applications to sentiment analysis. IEEE Transactions on Knowledge and Data Engineering 28, 2 (2016), 496--509.

Digital Library

[11]

Y. Ren, Y. Zhang, M. Zhang, and D. Ji. 2016. Improving twitter sentiment classification using topic-enriched multi-prototype word embeddings. In Proceedings of the Association for the Advancement of Artificial Intelligence. 3038--3044.

[12]

M. Lan, Z. Zhang, Y. Lu, and J. Wu. 2016. Three convolutional neural network based models for learning sentiment word vectors towards sentiment analysis. In Proceedings of the International Joint Conference on Neural Networks. 3172--3179.

[13]

Saif M. Mohammad and Peter D. Turney. 2013. NRC emotion lexicon. NRC Technical Report.

[14]

Carlo Strapparava and Alessandro Valitutti. 2004. Wordnet affect: An affective extension of wordnet. In Proceedings of the International Conference on Language Resources and Evaluation.

[15]

Soujanya Poria, Alexander Gelbukh, Erik Cambria, Amir Hussain, and Guang-Bin Huang. 2014. Emosenticspace: A novel framework for affective common-sense reasoning. Knowledge-Based Systems, 69:108--123.

Digital Library

[16]

Ameeta Agrawal and Aijun An. 2012. Unsupervised emotion detection from text using semantic and syntactic relations. In Proceedings of the International Conference on Web Intelligence and Intelligent Agent Technology.

[17]

Isidoros Perikos and Ioannis Hatzilygeroudis. 2016. Recognizing emotions in text using ensemble of classifiers. Engineering Applications of Artificial Intelligence, 51:191--201.

Digital Library

[18]

Munmun De Choudhury, Michael Gamon, and Scott Counts. 2012. Happy, nervous or surprised? Classification of human affective states in social media. In Proceedings of the International Conference on Web and Social Media.

[19]

Min-Ling Zhang and Zhi-Hua Zhou. 2014. A review on multi-label learning algorithms. IEEE Transactions on Knowledge and Data Engineering 26, 8 (2014), 1819--1837.

[20]

Xin Geng, Chao Yin, and Zhi-Hua Zhou. 2013 Facial age estimation by learning from label distributions. IEEE Transactions on Pattern Analysis and Machine Intelligence 35, 10 (2013), 2401--2412.

Digital Library

[21]

Yoon Kim. 2014. Convolutional neural networks for sentence classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.

[22]

Qiao Qian, Minlie Huang, and Xiaoyan Zhu. 2017. Linguistically regularized LSTMs for sentiment classification. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.

[23]

Carlo Strapparava and Rada Mihalcea. 2007. Semeval-2007 task 14: Affective text. In Proceedings of International Workshop on Semantic Evaluations.

[24]

R. Plutchik. 2005. Emotions and life: Perspectives from psychology, biology, and evolution. Psychology 8 Marketing. 97--101.

[25]

R. Plutchik. 2002. Emotions and life. Perspectives from Psychology Biology 8 Evolution.

[26]

S. Wang and X. Meng. 2018. Multi-emotion category improving embedding for sentiment classification. In Proceedings of the Conference on Information and Knowledge Management. 1719--1722.

[27]

J. Pennington, R. Socher, and C. D. Manning. 2014. GloVe: Global vectors forword representation. In Proceedings of the Empirical Methods of Natural Language Processing. 1532--1543.

[28]

G. A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM 38, 11 (1995), 39--41.

Digital Library

[29]

C. F. Baker, C. J. Fillmore, and J. B. Lowe. 1998. The Berkeley framenet project. In Proceedings of the Association for Computational Linguistics. 86--90.

[30]

J. Ganitkevitch, B. Van Durme, and C. Callison-Burch. 2013. PPDB: The paraphrase database. In Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics. 758--764.

[31]

E. Cambria. 2016. Affective computing and sentiment analysis. IEEE Intelligent Systems 31, 2 (2016), 102--107.

Digital Library

[32]

Yequan Wang, Minlie Huang, Xiaoyan Zhu, and Li Zhao. 2016. Attention-based LSTM for aspect-level sentiment classification. In Empirical Methods Natural Language Processing. 606--615

[33]

Y. Li, Q. Pan, T. Yang, et al. 2017. Learning word representation for sentiment analysis. Cognitive Computation. 9, 6 (2017), 843--851.

[34]

M. Munezero, C. S. Montero, E. Sutinen, and J. Pajunen. 2014. Are they different? affect, feeling, emotion, sentiment, and opinion detection in text. IEEE Transactions on Affective Computing 5, 2 (2014), 101--111.

[35]

M. B. Arnold. 1950. An excitatory theory of emotion. In Feelings and Emotions: The Mooseheart Symposium, M. L. Reymert (Ed.). McGraw-Hill, 11--33.

[36]

P. Ekman. 1992. An argument for basic emotions. Cognition 8 Emotion 6 (3–4), 169--200.

[37]

C. E. Izard. 1991. The Psychology of Emotions. Plenum, New York.

[38]

W. MacDougall. 1921. An Introduction to Social Psychology. John W. Luce, Boston, MA.

[39]

Saif M. Mohammad, Felipe Bravo-Marquez, Mohammad Salameh, and Svetlana Kiritchenko. 2018. Semeval-2018 Task 1: Affect in tweets. In Proceedings of International Workshop on Semantic Evaluation (SemEval-2018), New Orleans, LA.

Cited By

Liapis CKaranikola AKotsiantis S(2025)Enhancing sentiment analysis with distributional emotion embeddingsNeurocomputing10.1016/j.neucom.2025.129822(129822)Online publication date: Mar-2025
https://doi.org/10.1016/j.neucom.2025.129822
Parian-Scherb MUhrig PRossetto LDupont SSchuldt H(2024)Gesture retrieval and its application to the study of multimodal communicationInternational Journal on Digital Libraries10.1007/s00799-023-00367-025:4(585-601)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00799-023-00367-0
Park EStorey V(2023)Emotion Ontology Studies: A Framework for Expressing Feelings Digitally and its Application to Sentiment AnalysisACM Computing Surveys10.1145/355571955:9(1-38)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1145/3555719
Show More Cited By

Index Terms

Emo2Vec: Learning Emotional Embeddings via Multi-Emotion Category

Recommendations

Multi-Emotion Category Improving Embedding for Sentiment Classification
CIKM '18: Proceedings of the 27th ACM International Conference on Information and Knowledge Management

Sentiment analysis and opinion mining are significant and valuable for subject information extraction from the text. Word embedding that can map the words to low-dimensional vector representations has been widely used in natural language processing ...
Unsupervised Bilingual Sentiment Word Embeddings for Cross-lingual Sentiment Classification
ICIAI '20: Proceedings of the 2020 the 4th International Conference on Innovation in Artificial Intelligence

In recent years, bilingual word embeddings have been used to promote sentiment classification task in low-resource languages. However, existing bilingual word embedding methods either require annotated cross-lingual data or fail to capture enough ...
Semi-supervised Aspect-level Sentiment Classification Model based on Variational Autoencoder
Abstract
Aspect-level sentiment classification aims to predict the sentiment of a text in different aspects and it is a fine-grained sentiment analysis task. Recent work exploits an Attention-based Long Short-Term Memory Network to perform ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Internet Technology

ACM Transactions on Internet Technology Volume 20, Issue 2

Special Section on Emotions in Conflictual Social Interactions and Regular Papers

May 2020

256 pages

ISSN:1533-5399

EISSN:1557-6051

DOI:10.1145/3386441

Editor:
Ling Liu
Georgia Institute of Technology, USA

Issue’s Table of Contents

Copyright © 2020 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 April 2020

Accepted: 01 November 2019

Revised: 01 September 2019

Received: 01 March 2019

Published in TOIT Volume 20, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

National Key Research and Development Program of China
the Science and Technology Opening up Cooperation project of Henan Province
Fundamental Research Funds for the Central Universities
Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
955
Total Downloads

Downloads (Last 12 months)74
Downloads (Last 6 weeks)5

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liapis CKaranikola AKotsiantis S(2025)Enhancing sentiment analysis with distributional emotion embeddingsNeurocomputing10.1016/j.neucom.2025.129822(129822)Online publication date: Mar-2025
https://doi.org/10.1016/j.neucom.2025.129822
Parian-Scherb MUhrig PRossetto LDupont SSchuldt H(2024)Gesture retrieval and its application to the study of multimodal communicationInternational Journal on Digital Libraries10.1007/s00799-023-00367-025:4(585-601)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/s00799-023-00367-0
Park EStorey V(2023)Emotion Ontology Studies: A Framework for Expressing Feelings Digitally and its Application to Sentiment AnalysisACM Computing Surveys10.1145/355571955:9(1-38)Online publication date: 16-Jan-2023
https://dl.acm.org/doi/10.1145/3555719
Liu R(2023)Design of Robot Interactive Emotion Model Based on Artificial Intelligence Technology2023 International Conference on Power, Electrical Engineering, Electronics and Control (PEEEC)10.1109/PEEEC60561.2023.00142(709-714)Online publication date: 25-Sep-2023
https://doi.org/10.1109/PEEEC60561.2023.00142
Zhu ZMao K(2023)Knowledge-based BERT word embedding fine-tuning for emotion recognitionNeurocomputing10.1016/j.neucom.2023.126488552:COnline publication date: 1-Oct-2023
https://dl.acm.org/doi/10.1016/j.neucom.2023.126488
Aramanda AMd.Abdul SVedala R(2023)enemos-p: An enhanced emotion specific prediction for recommender systemsExpert Systems with Applications10.1016/j.eswa.2023.120190227(120190)Online publication date: Oct-2023
https://doi.org/10.1016/j.eswa.2023.120190
Aramanda AAbdul SVedala R(2021)Refining User Ratings Using User Emotions for Recommender SystemsThe 23rd International Conference on Information Integration and Web Intelligence10.1145/3487664.3487666(3-10)Online publication date: 29-Nov-2021
https://dl.acm.org/doi/10.1145/3487664.3487666
Drakopoulos GGiannoukou IMylonas PSioutas S(2020)A Graph Neural Network For Assessing The Affective Coherence Of Twitter Graphs2020 IEEE International Conference on Big Data (Big Data)10.1109/BigData50022.2020.9378492(3618-3627)Online publication date: 10-Dec-2020
https://doi.org/10.1109/BigData50022.2020.9378492

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Issue’s Table of Contents