Abstract
In this paper, we introduce an innovative approach that combines multi-modal sentiment analysis with mechanisms for safeguarding user information security. Our objective is to effectively combat harmful comments and precisely forecast trends in product sentiment. Initially, we detect and secure sensitive user information by implementing regular expressions to ensure user confidentiality and security. First, we apply statistical analysis and K-means++ algorithms to screen users who post malicious reviews. Next, we develop a novel multi-modal sentiment analysis and prediction model that incorporates the pre-trained BERT model and Swin Transformer model for feature extraction from comment text and image data. Furthermore, the expressive capability of image features is enhanced with the aid of the SENet model. We input the image features, improved by the SENet model, as well as the text features, extracted by the BERT model, into the Transformer model for fusion, and the classification probability is determined using the SoftMax function. We employ the Prophet method to combine sentiment indicators and time series features, which allows us to predict the upcoming sentiment trends of product reviews. The algorithm is implemented in the evaluation of the Amazon book review datasets and is compared to other algorithms such as Bert, yielding an accuracy of 94.15%. This study is of significant value for enabling real-time monitoring of product sentiment trends, filtering malicious reviews, and enhancing both product management and user experience.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Data Availability Statement
The data that support the findings of this study are available from the first author upon reasonable request.
References
Ringki, D., Thoudam, D.S.: Multimodal sentiment analysis: a survey of methods, trends, and challenges. ACM Comput. Surv. 55(13s), 1–38 (2023). Article No.: 270
Srinivasan, K., Raman, K., Chen, J., Bendersky, M., Najork, M., Pang, L.: Wit: Wikipedia-based image text dataset for multi-modal multilingual machine learning. In: Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2402–2406 (2021)
Meng, X., Wei, F., Liu, X., Zhou, M., Xu, G., Wang, H.: Cross-lingual mixture model for sentiment classification. In: Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics, Volume 1: Long Papers, pp. 572–581 (2012)
Xu, R., Xu, J., Wang, X.: Instance level transfer learning for cross lingual opinion analysis. In: Proceedings of the 2nd Workshop on Computational Approaches to Subjectivity and Sentiment Analysis (WASSA 2011), pp. 182–188 (2011)
Appel, O., Chiclana, F., Carter, J., et al.: A hybrid approach to the sentiment analysis problem at the sentence level. Knowl.-Based Syst. 108, 110–124 (2016)
Kim, S., Hovy, E.: Determining the sentiment of opinions. In: Proceedings of the 20th International Conference on Computational Linguistics, pp. 1367–1373 (2004)
Miller, G.A.: WordNet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)
Mohammad, S., Dunne, C., Dorr, B.: Generating high-coverage semantic orientation lexicons from overtly marked words and a thesaurus. In: Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing, pp. 599–608 (2009)
Taboada, M., Brooke, J., Toiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37(2), 267–307 (2011)
Pandey, P., Govilkar, S.: A framework for sentiment analysis in Hindi using HSWN. Int. J. Comput. Appl. 119(19) (2015)
Qiu, G., He, X., Zhang, F., Shi, Y., Bu, J., Chen, C.: DASA: dissatisfaction-oriented advertising based on sentiment. Expert Syst. Appl. 37(9), 6182–6191 (2010)
Nasukawa, T., Yi, J.: Sentiment analysis: capturing favorability using natural language processing. In: Proceedings of the 2nd International Conference on Knowledge Capture, pp. 70–77 (2003)
Turney, P.: Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, Philadelphia, Pennsylvania, USA, pp. 417–424. Association for Computational Linguistics (2002). https://doi.org/10.3115/1073083.1073153
Lu, Y., Kong, X., Quan, X., Liu, W., Xu, Y.: Exploring the sentiment strength of user reviews. In: Chen, L., Tang, C., Yang, J., Gao, Y. (eds.) WAIM 2010. LNCS, vol. 6184, pp. 471–482. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-14246-8_46
Dai, Z., Yang, Z., Yang, Y., et al.: Attentive Language Models beyond a Fixed-Length Context. arXiv preprint arXiv:1901.02860 (2019). https://doi.org/10.48550/arXiv.1901.02860
Chen, T., Yu, F.X., Chen, J., Cui, Y., Chen, Y.Y., Chang, S.F.: Object-based visual sentiment concept analysis and application. In: Proceedings of the 22nd ACM International Conference on Multimedia (MM 2014), pp. 367–376 (2014). Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2647868.2654935
Yuan, J., McDonough, S., You, Q., Luo, J.: Sentribute: image sentiment analysis from a mid-level perspective. In: Proceedings of the Second International Workshop on Issues of Sentiment Discovery and Opinion Mining, pp. 1–8 (2013)
Borth, D., Ji, R., Chen, T., Breuel, T., Chang, S.F.: Large-scale visual sentiment ontology and detectors using adjective noun pairs. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 223–232 (2013)
Borth, D., Chen, T., Ji, R., Chang, S.F.: Sentibank: large-scale ontology and classifiers for detecting sentiment and emotions in visual content. In: Proceedings of the 21st ACM International Conference on Multimedia, pp. 459–460 (2013)
Barbosa, L., Feng, J.: Robust sentiment detection on twitter from biased and noisy data. In: Coling 2010: Poster Volume, pp. 36–44 (2010)
Davidiv, D., Tsur, O., Rappoport, A.: Enhanced sentiment learning using twitter hashtags and smileys. In: Coling 2010: Poster Volume, pp. 23–27 (2010)
Go, A., Bhayani, R., Huang, L.: Twitter Sentiment Classification using Distant Supervision (2009)
You, Q., Luo, J., Jin, H., Yang, J.: Robust image sentiment analysis using progressively trained and domain transferred deep networks. In: Twenty-Ninth AAAI Conference on Artificial Intelligence (2015)
Campos, V., Jou, B., Giro-i Nieto, X.: From pixels to sentiment: fine-tuning CNNs for visual sentiment prediction. Image Vis. Comput. 65, 15–22 (2017)
Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up?: sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 Conference on Empirical Methods in Natural Language Processing, vol. 10, pp. 79–86. Association for Computational Linguistics (2002)
Poria, S., Cambria, E., Winterstein, G., Huang, G.-B.: Sentic patterns: dependency-based rules for concept-level sentiment analysis. Knowl.-Based Syst. 69, 45–63 (2014)
Bayer, M., Kaufhold, M.A., Reuter, C.: A survey on data augmentation for text classification. ACM Comput. Surv. 55(7), 1–39 (2022)
Tenney, I., Das, D., Pavlick, E.: BERT rediscovers the classical NLP pipeline. arXiv preprint arXiv:1905.05950 (2019)
You, Q., Cao, L., Jin, H., Luo, J.: Robust visual-textual sentiment analysis: when attention meets tree-structured recursive neural networks. In: Proceedings of the 24th ACM International Conference on Multimedia, pp. 1008–1017 (2016)
Wöllmer, M., et al.: YouTube movie reviews. Sentiment analysis in an audio-visual context. IEEE Intell. Syst. 28(3), 46–53 (2013)
Morency, L.-P., Mihalcea, R., Doshi, P.: Towards multimodal sentiment analysis: harvesting opinions from the web. In: Proceedings of the 13th International Conference on Multimodal Interfaces, pp. 169–176 (2011)
Pereira, M.H.R., Pádua, F.L.C., Pereira, A.C.M., Benevenuto, F., Dalip, D.H.: Fusing audio, textual, and visual features for sentiment analysis of news videos. In: Tenth International AAAI Conference on Web and Social Media (2016)
Poria, S., Cambria, E., Gelbukh, A.: Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 2539–2544 (2015)
Majumder, N., Hazarika, D., Gelbukh, A., Cambria, E., Poria, S.: Multimodal sentiment analysis using hierarchical fusion with context modeling. Knowl.-Based Syst. 161, 124–133 (2018)
Huang, F., Zhang, X., Zhao, Z., Xu, J., Li, Z.: Image-text sentiment analysis via deep multimodal attentive fusion. Knowl.-Based Syst. 167, 26–37 (2019)
Huddar, M.G., Sannakki, S.S., Rajpurohit, V.S.: Attention-based multimodal contextual fusion for sentiment and emotion classification using bidirectional LSTM. Multimedia Tools Appl. 80(9), 13059–13076 (2021)
Acknowledgments
The authors want to thank editors and unknown reviewers for providing useful suggestions. The authors also thank the Government of China to provide the required financial resources to complete the proposed project.
Funding
This research was funded by the National Natural Science Foundation of China (Grant No. 11901509), the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Grant No. 21KJB510009), National Nature Sciences Foundation of China with (Grant No. 42250410321) and the Natural Science Foundation of the Jiangsu Higher Education Institutions of China (Grant No. 19KJB110023).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Ethics declarations
The authors declare no conflict of interest.
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Chen, B., Jiang, L., Pan, X., Zhou, G., Sun, A., Li, D. (2024). Exploring Emotion Trends in Product Reviews: A Multi-modal Analysis with Malicious Comment Filtering and User Privacy Protection. In: Ge, C., Yung, M. (eds) Information Security and Cryptology. Inscrypt 2023. Lecture Notes in Computer Science, vol 14526. Springer, Singapore. https://doi.org/10.1007/978-981-97-0942-7_19
Download citation
DOI: https://doi.org/10.1007/978-981-97-0942-7_19
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-97-0941-0
Online ISBN: 978-981-97-0942-7
eBook Packages: Computer ScienceComputer Science (R0)