Skip to main content
SpringerLink
Log in

Enhancing Implicit Sentiment Learning via the Incorporation of Part-of-Speech for Aspect-Based Sentiment Analysis

  • Conference paper
  • First Online:
Chinese Computational Linguistics (CCL 2023)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 14232))

Included in the following conference series:

  • 259 Accesses

Abstract

Implicit sentiment modeling in aspect-based sentiment analysis is a challenging problem due to complex expressions and the lack of opinion words in sentences. Recent efforts focusing on implicit sentiment in ABSA mostly leverage the dependency between aspects and pretrain on extra annotated corpora. We argue that linguistic knowledge can be incorporated into the model to better learn implicit sentiment knowledge. In this paper, we propose a PLM-based, linguistically enhanced framework by incorporating Part-of-Speech (POS) for aspect-based sentiment analysis. Specifically, we design an input template for PLMs that focuses on both aspect-related contextualized features and POS-based linguistic features. By aligning with the representations of the tokens and their POS sequences, the introduced knowledge is expected to guide the model in learning implicit sentiment by capturing sentiment-related information. Moreover, we also design an aspect-specific self-supervised contrastive learning strategy to optimize aspect-based contextualized representation construction and assist PLMs in concentrating on target aspects. Experimental results on public benchmarks show that our model can achieve competitive and state-of-the-art performance without introducing extra annotated corpora.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 59.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 79.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://spacy.io/.

  2. 2.

    The statistics of parameters are derived from open-source repositories released by Yang and Li [57] and Li et al. [24].

References

  1. Baccianella, S., Esuli, A., Sebastiani, F., et al.: Sentiwordnet 3.0: an enhanced lexical resource for sentiment analysis and opinion mining. In: Lrec, vol. 10, pp. 2200–2204 (2010)

    Google Scholar 

  2. Brown, T., Mann, B., Ryder, N., Subbiah, M., Kaplan, J.D., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., et al.: Language models are few-shot learners. Adv. Neural. Inf. Process. Syst. 33, 1877–1901 (2020)

    Google Scholar 

  3. Cao, J., Liu, R., Peng, H., Jiang, L., Bai, X.: Aspect is not you need: No-aspect differential sentiment framework for aspect-based sentiment analysis. In: Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1599–1609. Association for Computational Linguistics, Seattle, United States, July 2022. https://doi.org/10.18653/v1/2022.naacl-main.115. https://aclanthology.org/2022.naacl-main.115

  4. Chen, Y.-C., et al.: UNITER: UNiversal image-TExt representation learning. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12375, pp. 104–120. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58577-8_7

    Chapter  Google Scholar 

  5. Chesley, P., Vincent, B., Xu, L., Srihari, R.K.: Using verbs and adjectives to automatically classify blog sentiment. Training 580(263), 233 (2006)

    Google Scholar 

  6. Dai, J., Yan, H., Sun, T., Liu, P., Qiu, X.: Does syntax matter? a strong baseline for aspect-based sentiment analysis with RoBERTa. In: Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1816–1829. Association for Computational Linguistics, Online, June 2021. https://doi.org/10.18653/v1/2021.naacl-main.146. https://aclanthology.org/2021.naacl-main.146

  7. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (Jun 2019). https://doi.org/10.18653/v1/N19-1423,https://aclanthology.org/N19-1423

  8. Dey, L., Haque, S.K.M.: Opinion mining from noisy text data. In: Proceedings of the Second Workshop on Analytics for Noisy Unstructured Text Data, AND 2008, pp. 83–90. Association for Computing Machinery, New York (2008). https://doi.org/10.1145/1390749.1390763. https://doi.org/10.1145/1390749.1390763

  9. Dong, L., Wei, F., Tan, C., Tang, D., Zhou, M., Xu, K.: Adaptive recursive neural network for target-dependent Twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 49–54. Association for Computational Linguistics, Baltimore, June 2014. https://doi.org/10.3115/v1/P14-2009,https://aclanthology.org/P14-2009

  10. Fan, S., et al.: Sentiment-aware word and sentence level pre-training for sentiment analysis. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 4984–4994. Association for Computational Linguistics, Abu Dhabi, December 2022. https://aclanthology.org/2022.emnlp-main.332

  11. Gao, T., Yao, X., Chen, D.: SimCSE: simple contrastive learning of sentence embeddings. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 6894–6910. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, November 2021. https://doi.org/10.18653/v1/2021.emnlp-main.552. https://aclanthology.org/2021.emnlp-main.552

  12. Gong, C., Yu, J., Xia, R.: Unified feature and instance based domain adaptation for aspect-based sentiment analysis. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 7035–7045 (2020)

    Google Scholar 

  13. Hadsell, R., Chopra, S., LeCun, Y.: Dimensionality reduction by learning an invariant mapping. In: 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 2006), vol. 2, pp. 1735–1742. IEEE (2006)

    Google Scholar 

  14. He, P., Gao, J., Chen, W.: Debertav 3: Improving deberta using electra-style pre-training with gradient-disentangled embedding sharing. arXiv preprint arXiv:2111.09543 (2021)

  15. Huang, B., Ou, Y., Carley, K.M.: Aspect level sentiment classification with attention-over-attention neural networks. In: Thomson, R., Dancy, C., Hyder, A., Bisgin, H. (eds.) SBP-BRiMS 2018. LNCS, vol. 10899, pp. 197–206. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-93372-6_22

    Chapter  Google Scholar 

  16. Jiang, T., et al.: PromptBERT: improving BERT sentence embeddings with prompts. In: Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pp. 8826–8837. Association for Computational Linguistics, Abu Dhabi, December 2022. https://aclanthology.org/2022.emnlp-main.603

  17. Ke, P., Ji, H., Liu, S., Zhu, X., Huang, M.: Sentilare: sentiment-aware language representation learning with linguistic knowledge. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6975–6988 (2020)

    Google Scholar 

  18. Kim, W., Son, B., Kim, I.: Vilt: vision-and-language transformer without convolution or region supervision. In: International Conference on Machine Learning, pp. 5583–5594. PMLR (2021)

    Google Scholar 

  19. Kiritchenko, S., Zhu, X., Cherry, C., Mohammad, S.: Nrc-canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 437–442 (2014)

    Google Scholar 

  20. Knight, P.A.: The sinkhorn-knopp algorithm: convergence and applications. SIAM J. Matrix Anal. Appl. 30(1), 261–275 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  21. Li, C., et al.: Sentiprompt: sentiment knowledge enhanced prompt-tuning for aspect-based sentiment analysis. arXiv preprint arXiv:2109.08306 (2021)

  22. Li, J., Chen, X., Hovy, E., Jurafsky, D.: Visualizing and understanding neural models in NLP. In: Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 681–691. Association for Computational Linguistics, San Diego, California, June 2016. https://doi.org/10.18653/v1/N16-1082. https://aclanthology.org/N16-1082

  23. Li, X., Bing, L., Lam, W., Shi, B.: Transformation networks for target-oriented sentiment classification. arXiv preprint arXiv:1805.01086 (2018)

  24. Li, Z., Zou, Y., Zhang, C., Zhang, Q., Wei, Z.: Learning implicit sentiment in aspect-based sentiment analysis with supervised contrastive pre-training. In: Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 246–256. Association for Computational Linguistics, Online and Punta Cana, Dominican Republic, November 2021. https://doi.org/10.18653/v1/2021.emnlp-main.22. https://aclanthology.org/2021.emnlp-main.22

  25. Liang, B., et al.: Enhancing aspect-based sentiment analysis with supervised contrastive learning. In: Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 3242–3247 (2021)

    Google Scholar 

  26. Liang, B., Su, H., Gui, L., Cambria, E., Xu, R.: Aspect-based sentiment analysis via affective knowledge enhanced graph convolutional networks. Knowl.-Based Syst. 235, 107643 (2022)

    Article  Google Scholar 

  27. Liu, Y., et al.: Roberta: a robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692 (2019)

  28. Loshchilov, I., Hutter, F.: Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101 (2017)

  29. Ma, D., Li, S., Zhang, X., Wang, H.: Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893 (2017)

  30. Ma, F., Zhang, C., Zhang, B., Song, D.: Aspect-specific context modeling for aspect-based sentiment analysis. In: Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24–25, 2022, Proceedings, Part I, pp. 513–526 (2022)

    Google Scholar 

  31. Ma, X., Gao, Y., Hu, Z., Yu, Y., Deng, Y., Hovy, E.: Dropout with expectation-linear regularization. arXiv preprint arXiv:1609.08017 (2016)

  32. Nicholls, C., Song, F.: Improving sentiment analysis with part-of-speech weighting. In: 2009 International Conference on Machine Learning and Cybernetics, vol. 3, pp. 1592–1597. IEEE (2009)

    Google Scholar 

  33. Paszke, A., et al.: Pytorch: an imperative style, high-performance deep learning library. In: Wallach, H., Larochelle, H., Beygelzimer, A., d’Alché-Buc, F., Fox, E., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 32. Curran Associates, Inc. (2019). https://proceedings.neurips.cc/paper_files/paper/2019/file/bdbca288fee7f92f2bfa9f7012727740-Paper.pdf

  34. Petrov, S., Das, D., McDonald, R.: A universal part-of-speech tagset. In: Proceedings of the Eighth International Conference on Language Resources and Evaluation (LREC’12). pp. 2089–2096. European Language Resources Association (ELRA), Istanbul, Turkey, May 2012. https://www.lrec-conf.org/proceedings/lrec2012/pdf/274_Paper.pdf

  35. Peyré, G., Cuturi, M., et al.: Computational optimal transport: with applications to data science. Found. Trends Mach. Learn. 11(5–6), 355–607 (2019)

    Google Scholar 

  36. Phan, M.H., Ogunbona, P.O.: Modelling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3211–3220. Association for Computational Linguistics, Online, July 2020. https://doi.org/10.18653/v1/2020.acl-main.293. https://aclanthology.org/2020.acl-main.293

  37. Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 27–35. Association for Computational Linguistics, Dublin, August 2014. https://doi.org/10.3115/v1/S14-2004. https://aclanthology.org/S14-2004

  38. Russo, I., Caselli, T., Strapparava, C.: Semeval-2015 task 9: clipeval implicit polarity of events. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp. 443–450 (2015)

    Google Scholar 

  39. Sennrich, R., Haddow, B., Birch, A.: Neural machine translation of rare words with subword units. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1715–1725 (2016)

    Google Scholar 

  40. Simonyan, K., Vedaldi, A., Zisserman, A.: Deep inside convolutional networks: visualising image classification models and saliency maps. In: Proceedings of the International Conference on Learning Representations (ICLR). ICLR (2014)

    Google Scholar 

  41. Song, Y., Wang, J., Jiang, T., Liu, Z., Rao, Y.: Attentional encoder network for targeted sentiment classification. arXiv preprint arXiv:1902.09314 (2019)

  42. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  43. Sun, K., Zhang, R., Mensah, S., Mao, Y., Liu, X.: Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5679–5688 (2019)

    Google Scholar 

  44. Vaserstein, L.N.: Markov processes over denumerable products of spaces, describing large systems of automata. Problemy Peredachi Informatsii 5(3), 64–72 (1969)

    MathSciNet  MATH  Google Scholar 

  45. Vaswani, A., et al.: Attention is all you need. Advances in neural information processing systems 30 (2017)

    Google Scholar 

  46. Wang, F., Liu, H.: Understanding the behaviour of contrastive loss. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2495–2504 (2021)

    Google Scholar 

  47. Wang, K., Shen, W., Yang, Y., Quan, X., Wang, R.: Relational graph attention network for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 3229–3238 (2020)

    Google Scholar 

  48. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp. 606–615 (2016)

    Google Scholar 

  49. Wolf, T., et al.: Transformers: state-of-the-art natural language processing. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pp. 38–45. Association for Computational Linguistics, Online, October 2020. https://www.aclweb.org/anthology/2020.emnlp-demos.6

  50. Wu, H., Zhang, Z., Shi, S., Wu, Q., Song, H.: Phrase dependency relational graph attention network for aspect-based sentiment analysis. Knowl.-Based Syst. 236, 107736 (2022)

    Article  Google Scholar 

  51. Wu, L., et al.: R-drop: Regularized dropout for neural networks. Adv. Neural. Inf. Process. Syst. 34, 10890–10905 (2021)

    Google Scholar 

  52. Wu, Y., et al.: Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144 (2016)

  53. Xu, H., Liu, B., Shu, L., Yu, P.: Bert post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1 (2019)

    Google Scholar 

  54. Xu, L., Pang, X., Wu, J., Cai, M., Peng, J.: Learn from structural scope: improving aspect-level sentiment analysis with hybrid graph convolutional networks. Neurocomputing 518, 373–383 (2023)

    Article  Google Scholar 

  55. Yan, H., Dai, J., Ji, T., Qiu, X., Zhang, Z.: A unified generative framework for aspect-based sentiment analysis. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 2416–2429 (2021)

    Google Scholar 

  56. Yan, Y., Li, R., Wang, S., Zhang, F., Wu, W., Xu, W.: Consert: a contrastive framework for self-supervised sentence representation transfer. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 5065–5075 (2021)

    Google Scholar 

  57. Yang, H., Li, K.: Improving implicit sentiment learning via local sentiment aggregation. arXiv e-prints pp. arXiv-2110 (2021)

    Google Scholar 

  58. Yin, D., Meng, T., Chang, K.W.: Sentibert: a transferable transformer-based architecture for compositional sentiment semantics. arXiv preprint arXiv:2005.04114 (2020)

  59. Zeng, B., Yang, H., Xu, R., Zhou, W., Han, X.: LCF: a local context focus mechanism for aspect-based sentiment classification. Appl. Sci. 9(16), 3389 (2019)

    Article  Google Scholar 

  60. Zhang, C., Li, Q., Song, D.: Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4568–4578 (2019)

    Google Scholar 

  61. Zhang, C., Li, Q., Song, D.: Syntax-aware aspect-level sentiment classification with proximity-weighted convolution network. In: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1145–1148 (2019)

    Google Scholar 

  62. Zhang, K., et al.: Incorporating dynamic semantics into pre-trained language model for aspect-based sentiment analysis. In: Findings of the Association for Computational Linguistics: ACL 2022, pp. 3599–3610. Association for Computational Linguistics, Dublin, May 2022. https://doi.org/10.18653/v1/2022.findings-acl.285. https://aclanthology.org/2022.findings-acl.285

  63. Zhao, P., Hou, L., Wu, O.: Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. Knowl.-Based Syst. 193, 105443 (2020)

    Article  Google Scholar 

  64. Zheng, Y., Li, X., Nie, J.Y.: Store, share and transfer: learning and updating sentiment knowledge for aspect-based sentiment analysis. Inf. Sci. 635, 151–168 (2023). https://doi.org/10.1016/j.ins.2023.03.102. https://www.sciencedirect.com/science/article/pii/S0020025523004279

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61976062).

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Guangdong University of Foreign Studies, Guangzhou, China

    Junlang Wang, Xia Li, Junyi He, Yongqiang Zheng & Junteng Ma

Authors
  1. Junlang Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xia Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Junyi He
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yongqiang Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Junteng Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xia Li .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Technology, Tsinghua University, Beijing, China

    Maosong Sun

  2. Harbin Institute of Technology, Harbin, China

    Bing Qin

  3. Fudan University, Shanghai, China

    Xipeng Qiu

  4. School of Computing and Information, Singapore Management University, Singapore, Singapore

    Jiang Jing

  5. Institute of Software, Chinese Academy of Sciences, Beijing, China

    Xianpei Han

  6. Beijing Language and Culture University, Beijing, China

    Gaoqi Rao

  7. Chinese Academy of Sciences, Institute of Automation, Beijing, China

    Yubo Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wang, J., Li, X., He, J., Zheng, Y., Ma, J. (2023). Enhancing Implicit Sentiment Learning via the Incorporation of Part-of-Speech for Aspect-Based Sentiment Analysis. In: Sun, M., et al. Chinese Computational Linguistics. CCL 2023. Lecture Notes in Computer Science(), vol 14232. Springer, Singapore. https://doi.org/10.1007/978-981-99-6207-5_24

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-6207-5_24

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-6206-8

  • Online ISBN: 978-981-99-6207-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation