Abstract
Social text is a kind of user-generated data on social media, which can reflect the opinions and emotions of netizens in their social activities. The study about emotion recognition for social text could help us clarify netizens’ emotional position on certain issues, products, or services to support opinion monitoring, marketing management and so on. But with the continuous improvement of information technology, the mode of emotional expression has become increasingly complex and diverse. For the sake of concise and comprehensive implications, social texts are usually very short and noisy, which are often accompanied with visual context and pictorial multi-grained semantics. Therefore, traditional research strategies based on plain text analysis have great limitations faced with those contents. To this end, we in this article focus on the semantic and emotional uncertainty in social texts through taking into account the textual content of social text and its related non-text information to improve emotion semantic representations for social text emotion recognition. Specifically, to realize the visual context modeling, we first leverage the visual information associated with social text as the contextual supplement to enhance the emotional semantics of short social text. Then, we model the semantics of social text with different granularity to fully mine the limited information of social text. After that, with the help of visual context enhancement and multi-grained semantic mining, we are capable of alleviating the limited expression of social text and its uncertainty of semantics and emotions. This could represent the emotion semantics of social text effectively for its emotion recognition. Finally, extensive experiments on the real multi-modal dataset demonstrate that our proposed method has promising results with high efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Abbasi, A., Chen, H., Salem, A.: Sentiment analysis in multiple languages: feature selection for opinion classification in web forums. ACM Trans. Inf. Syst. (TOIS) 26(3), 1–34 (2008)
Abbasi, A., France, S., Zhang, Z., Chen, H.: Selecting attributes for sentiment classification using feature relation networks. TKDE 23(3), 447–462 (2010)
Acheampong, F.A., Wenyu, C., Nunoo-Mensah, H.: Text-based emotion detection: advances, challenges, and opportunities. Eng. Rep. 2(7), e12189 (2020)
Alswaidan, N., Menai, M.E.B.: A survey of state-of-the-art approaches for emotion recognition in text. Knowl. Inf. Syst. 62(8), 1–51 (2020)
Bakshi, R.K., Kaur, N., Kaur, R., Kaur, G.: Opinion mining and sentiment analysis. In: 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom), pp. 452–455. IEEE (2016)
Basiri, M.E., Nemati, S., Abdar, M., Cambria, E., Acharya, U.R.: ABCDM: an attention-based bidirectional CNN-RNN deep model for sentiment analysis. Futur. Gener. Comput. Syst. 115, 279–294 (2021)
Chung, J., Gulcehre, C., Cho, K., Bengio, Y.: Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014)
Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018)
Dong, Z., Dong, Q., Hao, C.: Hownet and its computation of meaning. In: COLING 2010, pp. 53–56 (2010)
Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)
Go, A., Bhayani, R., Huang, L.: Twitter sentiment classification using distant supervision. CS224N project report, Stanford 1(12), 2009 (2009)
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)
Hu, M., Liu, B.: Mining and summarizing customer reviews. In: SIGKDD 2004, pp. 168–177 (2004)
Kouloumpis, E., Wilson, T., Moore, J.: Twitter sentiment analysis: the good the bad and the OMG! In: AAAI 2011, vol. 5 (2011)
Ku, L.W., Liang, Y.T., Chen, H.H., et al.: Opinion extraction, summarization and tracking in news and blog corpora. In: AAAI 2006, vol. 100107, pp. 1–167 (2006)
LeCun, Y., et al.: Backpropagation applied to handwritten zip code recognition. Neural Comput. 1, 541–551 (1989)
Lin, C., He, Y.: Joint sentiment/topic model for sentiment analysis. In: CIKM 2009, pp. 375–384 (2009)
Liu, K.L., Li, W.J., Guo, M.: Emoticon smoothed language models for twitter sentiment analysis. In: AAAI 2012, vol. 26 (2012)
Luo, L.: Network text sentiment analysis method combining LDA text representation and GRU-CNN. Pers. Ubiquitous Comput. 23(3), 405–412 (2018). https://doi.org/10.1007/s00779-018-1183-9
Mishne, G., De Rijke, M., et al.: Capturing global mood levels using blog posts. In: AAAI 2006, vol. 6, pp. 145–152 (2006)
Montavon, G., Orr, G.B., Müller, K.-R. (eds.): Neural Networks: Tricks of the Trade. LNCS, vol. 7700. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8
Pal, S., Ghosh, S., Nag, A.: Sentiment analysis in the light of LSTM recurrent neural networks. Int. J. Synth. Emot. (IJSE) 9(1), 33–39 (2018)
Ping, Q., Chen, C.: Video highlights detection and summarization with lag-calibration based on concept-emotion mapping of crowd-sourced time-sync comments. arXiv preprint arXiv:1708.02210 (2017)
Quan, X., Wang, Q., Zhang, Y., Si, L., Wenyin, L.: Latent discriminative models for social emotion detection with emotional dependency. ACM Trans. Inf. Syst. (TOIS) 34(1), 1–19 (2015)
Rakhlin, A.: Convolutional neural networks for sentence classification. GitHub (2016)
Schuster, M., Paliwal, K.K.: Bidirectional recurrent neural networks. IEEE Trans. Signal Process. 45(11), 2673–2681 (1997)
Seal, D., Roy, U.K., Basak, R.: Sentence-level emotion detection from text based on semantic rules. In: Tuba, M., Akashe, S., Joshi, A. (eds.) Information and Communication Technology for Sustainable Development. AISC, vol. 933, pp. 423–430. Springer, Singapore (2020). https://doi.org/10.1007/978-981-13-7166-0_42
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR, San Diego, CA, USA, May 7–9 (2015)
Song, X.Y., Zhao, Y., Jin, L.T., Sun, Y., Liu, T.: Research on the construction of sentiment dictionary based on word2vec. In: Proceedings of the 2018 International Conference on Algorithms, Computing and Artificial Intelligence, pp. 1–6 (2018)
Strapparava, C., Valitutti, A., et al.: Wordnet affect: an affective extension of wordnet. In: Lrec, vol. 4, p. 40. Citeseer (2004)
Tang, D.: Sentiment-specific representation learning for document-level sentiment analysis. In: ICDM 2015, pp. 447–452 (2015)
Tang, D., Wei, F., Qin, Bing, et al.: Building large-scale twitter-specific sentiment lexicon: a representation learning approach. In: COLING, pp. 172–182 (2014)
Tripathi, G., Singh, G.: Sentiment analysis approach based N-gram and KNN classifier. Int. J. Adv. Res. Comput. Sci. 9(3), 209–212 (2018)
Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Vateekul, P., Koomsubha, T.: A study of sentiment analysis using deep learning techniques on Thai Twitter data. In: 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE), pp. 1–6. IEEE (2016)
Wang, H., Chen, E., Liu, Q., Xu, Tong, et al.: A united approach to learning sparse attributed network embedding. In: ICDM, pp. 557–566. IEEE (2018)
Wang, H., et al.: MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: SIGKDD 2019, pp. 1064–1072 (2019)
Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: EMNLP 2016, pp. 606–615 (2016)
Xu, L., Lin, H.: Ontology-driven affective Chinese text analysis and evaluation method. In: Affective Computing & Intelligent Interaction, Second International Conference, Acii, Lisbon, Portugal, September (2007)
Yu, H., Hatzivassiloglou, V.: Towards answering opinion questions: separating facts from opinions and identifying the polarity of opinion sentences. In: EMNLP 2003, pp. 129–136 (2003)
Zagibalov, T., Carroll, J.A.: Automatic seed word selection for unsupervised sentiment classification of Chinese text. In: COLING 2008, pp. 1073–1080 (2008)
Acknowledgements
This research was supported by the National Key Research and Development Program of China (No. 2016YFB1000904), the National Natural Science Foundation of China (No. 61727809) and the Scientific Program of the Higher Education Institution of Xinjiang (No. XJEDU2016S067).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Cao, W. et al. (2021). Exploiting Visual Context and Multi-grained Semantics for Social Text Emotion Recognition. In: Fang, L., Chen, Y., Zhai, G., Wang, J., Wang, R., Dong, W. (eds) Artificial Intelligence. CICAI 2021. Lecture Notes in Computer Science(), vol 13069. Springer, Cham. https://doi.org/10.1007/978-3-030-93046-2_66
Download citation
DOI: https://doi.org/10.1007/978-3-030-93046-2_66
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-93045-5
Online ISBN: 978-3-030-93046-2
eBook Packages: Computer ScienceComputer Science (R0)