Abstract
We propose and test multiple neuro-symbolic methods for sentiment analysis. They combine deep neural networks – transformers and recurrent neural networks – with external knowledge bases. We show that for simple models, adding information from knowledge bases significantly improves the quality of sentiment prediction in most cases. For medium-sized sets, we obtain significant improvements over state-of-the-art transformer-based models using our proposed methods: Tailored KEPLER and Token Extension. We show that the cases with the improvement belong to the hard-to-learn set.
This work was funded by the National Science Centre, Poland, project no. 2021/41/B/ST6/04471 (PK) and 2019/33/B/HS2/02814 (MP); the Polish Ministry of Education and Science, CLARIN-PL; the European Regional Development Fund as a part of the 2014–2020 Smart Growth Operational Programme, CLARIN – Common Language Resources and Technology Infrastructure, project number POIR.04.02.00-00C002/19; the statutory funds of the Department of Artificial Intelligence, Wrocław University of Science and Technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Triples: lemma, Part of Speech (PoS) and sense identifier.
- 2.
- 3.
Joy, fear, surprise, sadness, disgust, anger, trust and anticipation.
- 4.
Utility, truth, knowledge, beauty, happiness, futility, harm, ignorance, error, ugliness.
- 5.
- 6.
- 7.
References
plWordNet 4.5 (2021). http://hdl.handle.net/11321/834. CLARIN-PL
Al-Moslmi, T., Omar, N., Abdullah, S., Albared, M.A.: Approaches to cross-domain sentiment analysis: systematic lit. Review. IEEE Access 5, 16173–16192 (2017)
Augustyniak, L., Kajdanowicz, T., Kazienko, P., Kulisiewicz, M., Tuliglowicz, W.: An approach to sentiment analysis of movie reviews: lexicon based vs. classification. In: Polycarpou, M., de Carvalho, A.C.P.L.F., Pan, J.-S., Woźniak, M., Quintian, H., Corchado, E. (eds.) HAIS 2014. LNCS (LNAI), vol. 8480, pp. 168–178. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07617-1_15
Augustyniak, Ł., et al.: Simpler is better? Lexicon-based ensemble sentiment classification beats supervised methods. In: ASONAM 2014, pp. 924–929 (2014)
Bassignana, E., Basile, V., Patti, V.: Hurtlex: a multilingual lexicon of words to hurt. In: CLiC-it 2018, vol. 2253, pp. 1–6. CEUR-WS (2018)
Bojanowski, P., Grave, E., Joulin, A., Mikolov, T.: Enriching word vectors with subword information (2017)
Dziob, A., Piasecki, M., Rudnicka, E.: plWordNet 4.1 - a linguistically motivated, corpus-based bilingual resource. In: The 10th Global Wordnet Conference, pp. 353–362. Global Wordnet Association, July 2019
Ghosal, D., Hazarika, D., Roy, A., Majumder, N., Mihalcea, R., Poria, S.: Kingdom: knowledge-guided domain adaptation for sentiment analysis. arXiv:2005.00791 (2020)
Hripcsak, G., Rothschild, A.: Agreement, the f-measure, and reliability in information retrieval. J. Am. ER. Med. Inform. Ass. (JAMIA) 12(3), 296–298 (2005)
Janz, A., Piasecki, M.: A weakly supervised word sense disambiguation for polish using rich lexical resources. Poznan Stud. Cont. Ling. 55(2), 339–365 (2019)
Joseph, J., Vineetha, S., Sobhana, N.: A survey on deep learning based sentiment analysis. Mater. Today Proc. 58, 456–460 (2022)
Kanclerz, K., Miłkowski, P., Kocoń, J.: Cross-lingual deep neural transfer learning in sentiment analysis. Procedia Comput. Sci. 176, 128–137 (2020)
Ke, P., Ji, H., Liu, S., Zhu, X., Huang, M.: SentiLARE: sentiment-aware language representation learning with linguistic knowledge. arXiv:1911.02493 (2020)
Kocoń, J., Gawor, M.: Evaluating KGR10 Polish word embeddings in the recognition of temporal expressions using BiLSTM-CRF. Schedae Informaticae 27 (2018)
Kocoń, J., Miłkowski, P., Zaśko-Zielińska, M.: Multi-level sentiment analysis of PolEmo 2.0: extended corpus of multi-domain consumer reviews. In: CoNLL2019, pp. 980–991. ACL, November 2019
Koufakou, A., Pamungkas, E.W., Basile, V., Patti, V.: HurtBERT: incorporating lexical features with BERT for the detection of abusive language. In: The 4th Workshop on Online Abuse and Harms, pp. 34–43. ACL, November 2020
Ma, Y., Peng, H., Cambria, E.: Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In: AAAI 2018, vol. 32 (2018)
Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS 2013, pp. 3111–3119 (2013)
Kocoń, J., Miłkowski, P., Kanclerz, K.: MultiEmo: multilingual, multilevel, multidomain sentiment analysis corpus of consumer reviews. In: Paszynski, M., Kranzlmüller, D., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds.) ICCS 2021. LNCS, vol. 12743, pp. 297–312. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-77964-1_24
Plutchik, R.: EMOTION: A Psychoevolutionary Synthesis. Harper & Row (1980)
Puzynina, J.: Jȩzyk wartości [The language of values]. Polish Scientific Publishers PWN (1992)
Sun, Z., Deng, Z.H., Nie, J.Y., Tang, J.: RotatE: knowledge graph embedding by relational rotation in complex space. In: The International Conference on Learning Representations (ICLR) (2019)
Swayamdipta, S., et al.: Dataset cartography: mapping and diagnosing datasets with training dynamics. In: EMNLP 2020, pp. 9275–9293. ACL, November 2020
Tian, H., et al.: SKEP: sentiment knowledge enhanced pre-training for sentiment analysis (2020)
Vizcarra, J., Kozaki, K., Torres Ruiz, M., Quintero, R.: Knowledge-based sentiment analysis and visualization on social networks. NGC 39(1), 199–229 (2021)
Wang, X., Gao, T., Zhu, Z., Liu, Z., Li, J.Z., Tang, J.: KEPLER: a unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguist. 9, 176–194 (2021)
Zaśko-Zielińska, M., Piasecki, M.: Towards emotive annotation in plWordNet 4.0. In: The 9th Global Wordnet Conference, pp. 153–162. Global WordNet Association (2018)
Zhang, T., Wu, F., Katiyar, A., Weinberger, K.Q., Artzi, Y.: Revisiting few-sample BERT fine-tuning. arXiv:2006.05987 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kocoń, J. et al. (2022). Neuro-Symbolic Models for Sentiment Analysis. In: Groen, D., de Mulatier, C., Paszynski, M., Krzhizhanovskaya, V.V., Dongarra, J.J., Sloot, P.M.A. (eds) Computational Science – ICCS 2022. ICCS 2022. Lecture Notes in Computer Science, vol 13351. Springer, Cham. https://doi.org/10.1007/978-3-031-08754-7_69
Download citation
DOI: https://doi.org/10.1007/978-3-031-08754-7_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-08753-0
Online ISBN: 978-3-031-08754-7
eBook Packages: Computer ScienceComputer Science (R0)