skip to main content
note

Fusion Pre-trained Emoji Feature Enhancement for Sentiment Analysis

Published: 25 March 2023 Publication History

Abstract

Emoji are often used in social media to enrich users’ emotions, and they play an important role in the task of social media sentiment analysis. In practice, researchers are more likely to consider emoji as special symbols and treat them separately from the text. Some existing methods use emoji as a dictionary for matching or converting emoji into text. However, these methods disregard the relationship between emoji and context, blue and they do not reflect the emotions that users are expected to express. It is challenging to incorporate the original emotions of emoji in social media sentiment analysis. In this article, we propose the EPE model: Emoji Pre-trained feature Enhanced sentiment analysis. Specifically, we collected 8 million tweets and selected 5 million tweets with pre-trained emoji with context using the BERT model. We labeled 20,000 tweets as a three-category dataset and used Bi-LSTM with an attention layer to extract text features. Emoji were retained as key emotion information and combined with text features in the final layer as a connected vector for final prediction. Experimental results with our dataset showed that the proposed EPE model achieved better performance than other baseline models.

References

[1]
Sadam Al-Azani and El-Sayed El-Alfy. 2018. Emojis-based sentiment classification of arabic microblogs using deep recurrent neural networks. In Proceedings of the International Conference on Computing Sciences and Engineering. 1–6.
[2]
Francesco Barbieri, Luis Espinosa Anke, José Camacho-Collados, Steven Schockaert, and Horacio Saggion. 2018. Interpretable emoji prediction via label-wise attention LSTMs. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 4766–4771.
[3]
Naomi S. Baron. 2009. The myth of impoverished signal: Dispelling the spoken language fallacy for emoticons in online communication. In Proceedings of the Electronic Emotion: The Mediation of Emotion via Information and Communication Technologies. 107–135.
[4]
Peng Chen, Zhongqian Sun, Lidong Bing, and Wei Yang. 2017. Recurrent attention network on memory for aspect sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, 452–461.
[5]
Yuxiao Chen, Jianbo Yuan, Quanzeng You, and Jiebo Luo. 2018. Twitter sentiment analysis via bi-sense emoji embedding and attention-based LSTM. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference. ACM, 117–125.
[6]
Yuxiao Chen, Jianbo Yuan, Quanzeng You, and Jiebo Luo. 2018. Twitter sentiment analysis via bisense emoji embedding and attention-based LSTM. In Proceedings of the 2018 ACM Multimedia Conference on Multimedia Conference. 117–125.
[7]
Zhenpeng Chen, Sheng Shen, Ziniu Hu, Xuan Lu, Qiaozhu Mei, and Xuanzhe Liu. 2019. Emoji-powered representation learning for cross-lingual sentiment classification. In Proceedings of the World Wide Web Conference, WWW 2019. ACM, 251–262.
[8]
Dmitry Davidov, Oren Tsur, and Ari Rappoport. 2010. Enhanced sentiment learning using twitter hashtags and smileys. In Proceedings of the COLING 2010, 23rd International Conference on Computational Linguistics. Chinese Information Processing Society of China, 241–249.
[9]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019. Association for Computational Linguistics, 4171–4186.
[10]
Bjarke Felbo, Alan Mislove, Anders Søgaard, Iyad Rahwan, and Sune Lehmann. 2017. Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017. Association for Computational Linguistics, 1615–1625.
[11]
Xia Hu, Jiliang Tang, Huiji Gao, and Huan Liu. 2013. Unsupervised sentiment analysis with emotional signals. In Proceedings of the 22nd International World Wide Web Conference. International World Wide Web Conferences Steering Committee/ACM, 607–618.
[12]
Aditya Joshi, Pushpak Bhattacharyya, and Mark James Carman. 2017. Automatic sarcasm detection: A survey. ACM Computing Surveys 50, 5 (2017), 73:1–73:22.
[13]
Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014. The Association for Computer Linguistics, 655–665.
[14]
Svetlana Kiritchenko, Xiaodan Zhu, and Saif M. Mohammad. 2014. Sentiment analysis of short informal texts. Journal of Artificial Intelligence Research 50, 2014 (2014), 723–762.
[15]
Tuan Anh Le, David Moeljadi, Yasuhide Miura, and Tomoko Ohkuma. 2016. Sentiment analysis for low resource languages: A study on informal indonesian tweets. In Proceedings of the 12th Workshop on Asian Language Resources.Kôiti Hasida, Kam-Fai Wong, Nicoletta Calzorari, and Key-Sun Choi (Eds.), The COLING 2016 Organizing Committee, 123–131.
[16]
Da Li, Rafal Rzepka, Michal Ptaszynski, and Kenji Araki. 2018. Emoticon-aware recurrent neural network model for chinese sentiment analysis. In Proceedings of the 9th International Conference on Awareness Science and Technology. IEEE, 161–166.
[17]
Da Li, Rafal Rzepka, Michal Ptaszynski, and Kenji Araki. 2019. Emoji-aware attention-based bi-directional GRU network model for chinese sentiment analysis. In Joint Proceedings of the Workshops on Linguistic and Cognitive Approaches to Dialog Agents (LaCATODA 2019) and on Bridging the Gap Between Human and Automated Reasoning (BtG 2019) co-located with 28th International Joint Conference on Artificial Intelligence (IJCAI 2019).CEUR-WS.org, 11–18.
[18]
Da Li, Rafal Rzepka, Michal Ptaszynski, and Kenji Araki. 2019. A novel machine learning-based sentiment analysis method for chinese social media considering chinese slang lexicon and emoticons. In Proceedings of the 2nd Workshop on Affective Content Analysis (AffCon 2019) Co-located with 33rd AAAI Conference on Artificial Intelligence . CEUR-WS.org, 88–103.
[19]
Bing Liu. 2012. Sentiment Analysis and Opinion Mining. Morgan and Claypool Publishers.
[20]
Yinxia Lou, Yue Zhang, Fei Li, Tao Qian, and Donghong Ji. 2020. Emoji-based sentiment analysis using attention networks. ACM Transactions on Asian and Low-Resource Language Information Processing 19, 5 (2020), 64:1–64:13.
[21]
Saif M. Mohammad, Svetlana Kiritchenko, and Xiaodan Zhu. 2013. NRC-Canada: Building the state-of-the-art in sentiment analysis of tweets. In Proceedings of the 7th International Workshop on Semantic Evaluation. The Association for Computer Linguistics, 321–327.
[22]
Ilaria Moschini. 2016. The “face with tears of joy” emoji. a socio-semiotic and multimodal insight into a japan-america mash-up. HERMES-Journal of Language and Communication in Business 55, 2016 (2016), 11–25.
[23]
Sascha Narr, Michael Hulfenhaus, and Sahin Albayrak. 2012. Language-independent twitter sentiment analysis. In Proceedings of the Learning, Knowledge, and Adaption Conference. 12–14.
[24]
Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up? Sentiment classification using machine learning techniques. In Proceedings of the 2002 Conference on Empirical Methods in Natural Language Processing. 79–86.
[25]
Yafeng Ren, Donghong Ji, and Han Ren. 2018. Context-augmented convolutional neural networks for twitter sarcasm detection. Neurocomputing 308, 2018 (2018), 1–7.
[26]
Jayashree Subramanian, Varun Sridharan, Kai Shu, and Huan Liu. 2019. Exploiting emojis for sarcasm detection. In Proceedings of the Social, Cultural, and Behavioral Modeling—12th International Conference, SBP-BRiMS 2019. Springer, 70–80.
[27]
Mike Thelwall, Kevan Buckley, and Georgios Paltoglou. 2012. Sentiment strength detection for the social web. Journal of the Association for Information Science and Technology 63, 1 (2012), 163–173.
[28]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical attention networks for document classification. In Proceedings of the NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. The Association for Computational Linguistics, 1480–1489.
[29]
Wen-tau Yih, Xiaodong He, and Christopher Meek. 2014. Semantic parsing for single-relation question answering. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics, 643–648.
[30]
Peng Zhou, Wei Shi, Jun Tian, Zhenyu Qi, Bingchen Li, Hongwei Hao, and Bo Xu. 2016. Attention-based bidirectional long short-term memory networks for relation classification. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. The Association for Computer Linguistics.

Cited By

View all
  • (2024)EEBERT: An Emoji-Enhanced BERT Fine-Tuning on Amazon Product Reviews for Text Sentiment ClassificationIEEE Access10.1109/ACCESS.2024.345603912(131954-131967)Online publication date: 2024
  • (2024)TaneNet: Two-Level Attention Network Based on Emojis for Sentiment AnalysisIEEE Access10.1109/ACCESS.2024.341637912(86106-86119)Online publication date: 2024
  • (2024)Comprehensive review and comparative analysis of transformer models in sentiment analysisKnowledge and Information Systems10.1007/s10115-024-02214-366:12(7305-7361)Online publication date: 1-Dec-2024
  • Show More Cited By

Index Terms

  1. Fusion Pre-trained Emoji Feature Enhancement for Sentiment Analysis

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Transactions on Asian and Low-Resource Language Information Processing
    ACM Transactions on Asian and Low-Resource Language Information Processing  Volume 22, Issue 4
    April 2023
    682 pages
    ISSN:2375-4699
    EISSN:2375-4702
    DOI:10.1145/3588902
    Issue’s Table of Contents

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 25 March 2023
    Online AM: 29 December 2022
    Accepted: 19 December 2022
    Revised: 21 September 2022
    Received: 01 September 2021
    Published in TALLIP Volume 22, Issue 4

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. Sentiment analysis
    2. social media
    3. emoji
    4. fusion pre-trained
    5. feature combination

    Qualifiers

    • Note

    Funding Sources

    • National Natural Science Foundation of China
    • China Scholarship Council
    • Natural Science Foundation for the Higher Education Institutions of Anhui Province of China

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)221
    • Downloads (Last 6 weeks)16
    Reflects downloads up to 18 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)EEBERT: An Emoji-Enhanced BERT Fine-Tuning on Amazon Product Reviews for Text Sentiment ClassificationIEEE Access10.1109/ACCESS.2024.345603912(131954-131967)Online publication date: 2024
    • (2024)TaneNet: Two-Level Attention Network Based on Emojis for Sentiment AnalysisIEEE Access10.1109/ACCESS.2024.341637912(86106-86119)Online publication date: 2024
    • (2024)Comprehensive review and comparative analysis of transformer models in sentiment analysisKnowledge and Information Systems10.1007/s10115-024-02214-366:12(7305-7361)Online publication date: 1-Dec-2024
    • (2023)HKG: A Novel Approach for Low Resource Indic Languages to Automatic Knowledge Graph ConstructionACM Transactions on Asian and Low-Resource Language Information Processing10.1145/3611306Online publication date: 2-Aug-2023

    View Options

    Login options

    Full Access

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    HTML Format

    View this article in HTML Format.

    HTML Format

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media