Skip to main content
SpringerLink
Log in

Learning for target-dependent sentiment based on local context-aware embedding

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Target-dependent sentiment classification is a fine-grained task of natural language processing to analyze the sentiment polarity of the targets. In order to address the difficulty of locating important sentiment information of targeted sentiment classification, recent research mostly applies attention mechanisms to capture the information of important context words, while the attention mechanism is subject to many drawbacks, e.g., dependent on network architecture and expensive. Recent studies show the significant effect of the local context focus (LCF) mechanism in capturing the relatedness between a target’s sentiment and its local context. However, the LCF simply applies the fusion of global and local context features to classify sentiment, neglecting to empower the network to be aware of deep information of local context. In this paper, we propose a novel local context-aware network (LCA-Net) based on the local context embedding (LCE). Moreover, accompanied by the sentiment classification loss, the local context prediction (LCP) loss is proposed to enhance the LCE. The experimental results on three commonly used datasets, i.e., the Laptop and Restaurant datasets from SemEval-2014 and a Twitter social dataset, show that all the LCA-Net variants achieve promising performance improvement compared to existing approaches in extracting local context features. Besides, we implement the LCA-Net with different neural networks, validating the transferability of LCA architecture.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. Both datasets are available at http://alt.qcri.org/semeval2014/task4.

  2. To obtain the same dimension of word representations, we conduct truncating and padding for the sentence.

  3. For a fair comparison of the improvement of the LCA-Net, the basic BERT was adopted to build LCA-BERT. We implement our models based on https://github.com/huggingface/transformers. And all the experiments are conducted on the RTX 2080 GPU.

  4. There is no domain-adapted BERT for the Twitter dataset, we employ the Restaurant domain-adapted BERT, instead.

  5. The datasets can be found at http://alt.qcri.org/semeval2014/task4.

References

  1. Su Y-J, Hu W-C, Jiang J-H, Su R-Y (2020) A novel LMAEB-CNN model for Chinese microblog sentiment analysis. J Supercomput 1–15

  2. Balakrishnan V, Lok PY, Rahim HA (2020) A semi-supervised approach in detecting sentiment and emotion based on digital payment reviews. J Supercomput 1–16

  3. Ray B, Garain A, Sarkar R (2021) An ensemble-based hotel recommender system using sentiment analysis and aspect categorization of hotel reviews. Appl Soft Comput 98:106935

    Article  Google Scholar 

  4. Liu B (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167

    Article  Google Scholar 

  5. Li W, Zhu L, Shi Y, Guo K, Cambria E (2020) User reviews: sentiment analysis using lexicon integrated two-channel CNN-LSTM family model. Appl Soft Comput 94:106435

    Article  Google Scholar 

  6. Cambria E (2016) Affective computing and sentiment analysis. IEEE Intell Syst 31(2):102–107

    Article  Google Scholar 

  7. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiouc H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), Dublin, Ireland, August 2014. Association for Computational Linguistics, pp 27–35

  8. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) SemEval-2015 task 12: aspect based sentiment analysis. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), Denver, Colorado, June 2015. Association for Computational Linguistics, pp 486–495

  9. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, AL-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra SM, Eryiğit G (2016) SemEval-2016 task 5: aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, California, June 2016. Association for Computational Linguistics, pp 19–30

  10. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate, 2014

  11. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Texas, November 2016. Association for Computational Linguistics, pp 606–615

  12. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp 4068–4074

  13. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3433–3442

  14. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN , Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in Neural Information Processing Systems, pp 5998–6008

  15. Yang H, Zeng B, Yang J, Song Y, Ruyang X (2021) A multi-task learning model for Chinese-oriented aspect polarity classification and aspect term extraction. Neurocomputing 419:344–356

    Article  Google Scholar 

  16. Phan MH, Ogunbona PO (2020) Modelling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 3211–3220

  17. Luo W, Yi S, Chen J, Weng S, Dong Z (2020) Does ensemble really work when facing the twitter semantic classification? In: 2020 5th International Conference on Computational Intelligence and Applications (ICCIA). IEEE, pp 39–43

  18. Zeng B, Yang H, Ruyang X, Zhou W, Han X (2019) LCF: a local context focus mechanism for aspect-based sentiment classification. Appl Sci 9(16):3389

    Article  Google Scholar 

  19. Devlin J, Chang M-W, Lee K, Toutanova K (2019) Bert: Pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp 4171–4186

  20. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) NRC-Canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp 437–442

  21. Wagner J, Arora P, Vaíllo SC, Barman U, Bogdanova D, Foster J, Tounsi L (2014) DCU: Aspect-based polarity classification for SemEval task 4. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp 223–229

  22. Vo D-T, Zhang Y (2015) Target-dependent twitter sentiment classification with rich automatic features. In: Twenty-Fourth International Joint Conference on Artificial Intelligence. AAAI Press, pp 1347–1353

  23. Tang D, Qin B, Feng X, Liu T (2016) Effective LSTMs for target-dependent sentiment classification. In: Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, Osaka, Japan, December 2016. The COLING 2016 Organizing Committee, pp 3298–3307

  24. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pp 452–461

  25. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 946–956

  26. Mao Q, Li J, Wang S, Zhang Y, Peng H, He M, Wang L (2019) Aspect-based sentiment classification with attentive neural turing machines. In: IJCAI, pp 5139–5145

  27. Du C, Sun H, Wang J, Qi Q, Liao J, Xu T, Liu M (2019) Capsule network with interactive attention for aspect-level sentiment classification. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 5492–5501

  28. Sun K, Zhang R, Mensah S, Mao Y, Liu X (2019) Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 5683–5692

  29. Lin P, Yang M , Lai J (2019) Deep mask memory network with semantic dependency and context moment for aspect level sentiment classification. In: IJCAI, pp 5088–5094

  30. Liu N, Shen B (2020) Aspect-based sentiment analysis with gated alternate neural network. Knowl Based Syst 188:105010

    Article  Google Scholar 

  31. Zhao P, Hou L, Ou W (2020) Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. Knowl Based Syst 193:105443

    Article  Google Scholar 

  32. Huang J, Meng Y, Guo F, Ji H, Han J (2020) Aspect-based sentiment analysis by aspect-sentiment joint embedding. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 6989–6999

  33. Shuang K, Yang Q, Loo J, Li R, Gu M (2020) Feature distillation network for aspect-based sentiment analysis. Inf Fusion

  34. Chen Z, Qian T (2020) Relation-aware collaborative learning for unified aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 3685–3694

  35. Young T, Cambria E, Chaturvedi I, Huang M, Zhou H, Biswas S (2017) Augmenting end-to-end dialog systems with commonsense knowledge. arXiv preprint arXiv:1709.05453

  36. Rietzler A, Stabinger S, Opitz P, Engl S (2019) Adapt or get left behind: domain adaptation through bert language model finetuning for aspect-target sentiment classification. arXiv preprint arXiv:1908.11860

  37. Zhang B, Li X, Xiaofei X, Leung K-C, Chen Z, Ye Y (2020) Knowledge guided capsule attention network for aspect-based sentiment analysis. IEEE/ACM Trans Audio Speech Lang Process 28:2538–2551

    Article  Google Scholar 

  38. Zhou J, Huang JX, Hu QV, He L (2020) SK-GCN: Modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification. Knowl Based Syst 205:106292

  39. Lan Z, Chen M, Goodman S, Gimpel K, Sharma P, Soricut R (2019) Albert: a lite bert for self-supervised learning of language representations. In: International Conference on Learning Representations

  40. Sun C, Huang L, Qiu X (2019) Utilizing bert for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp 380–385

  41. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Targeted sentiment classification with attentional encoder network. In: International Conference on Artificial Neural Networks. Springer, Berlin, pp 93–103

  42. Phan MH, Ogunbona PO (2020) Modeling context and syntactical features for aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp 3211–3220

  43. Pennington J, Socher R, Manning CD (2014) Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 1532–1543

  44. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (volume 2: Short papers), pp 49–54

  45. Liu J, Zhang Y (2017) Attention modeling for targeted sentiment. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pp 572–577

  46. Xu H, Liu B, Shu L, Yu P (2019) BERT post-training for review reading comprehension and aspect-based sentiment analysis. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), Minneapolis, Minnesota, June 2019. Association for Computational Linguistics, pp 2324–2335

Download references

Acknowledgements

Thanks to the anonymous reviewers and the scholars who helped us. This research is funded by the National Natural Science Foundation of China, project approval number: 61876067; The Guangdong General Colleges and Universities Special Projects in Key Areas of Artificial Intelligence of China, project number: 2019KZDZX1033. And this research is supported by the Innovation Project of Graduate School of South China Normal University, project number: 2019LKXM038.

Author information

Authors and Affiliations

  1. School of Software, South China Normal University, Foshan, 528225, China

    Biqing Zeng, Shuai Liu & Mayi Xu

  2. School of Computer, South China Normal University, Guangzhou, 510631, China

    Heng Yang

Authors
  1. Biqing Zeng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Heng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shuai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Mayi Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Heng Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zeng, B., Yang, H., Liu, S. et al. Learning for target-dependent sentiment based on local context-aware embedding. J Supercomput 78, 4358–4376 (2022). https://doi.org/10.1007/s11227-021-04047-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04047-1

Keywords

Search

Navigation