Skip to main content

Advertisement

Springer Nature Link
Log in

Graph convolutional networks with hierarchical multi-head attention for aspect-level sentiment classification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Aspect-level sentiment classification has been widely used by researchers as a fine-grained sentiment classification task to predict the sentiment polarity of specific aspect words in a given sentence. Previous studies have shown relatively good experimental results using graph convolutional networks, so more and more approaches are beginning to exploit sentence structure information for this task. However, these methods do not link aspect word and context well. To address this problem, we propose a method that utilizes a hierarchical multi-head attention mechanism and a graph convolutional network (MHAGCN). It fully considers syntactic dependencies and combines semantic information to achieve interaction between aspect words and context. To fully validate the effectiveness of the method proposed in this paper, we conduct extensive experiments on three benchmark datasets, which, according to the experimental results, show that the method outperforms current methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. https://spacy.io/.

References

  1. Kang H, Yoo SJ, Han D (2012) Senti-lexicon and improved naïve bayes algorithms for sentiment analysis of restaurant reviews. Expert Syst. Appl. 39(5):6000–6010. https://doi.org/10.1016/j.eswa.2011.11.107

    Article  Google Scholar 

  2. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S(2014) SemEval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014), pp. 27– 35. Association for Computational Linguistics, Dublin, Ireland

  3. Schouten K, Frasincar F (2016) Survey on aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng. 28(3):813–830. https://doi.org/10.1109/TKDE.2015.2485209

    Article  Google Scholar 

  4. Tay Y, Tuan L.A, Hui S.C(2018) Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: McIlraith, S.A., Weinberger, K.Q. (eds.) Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence,(AAAI-18), pp. 5956– 5963

  5. Tang D, Qin B, Liu T(2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1422– 1432. Association for Computational Linguistics, Lisbon, Portugal

  6. Zainuddin N, Selamat A, Ibrahim R (2018) Hybrid sentiment classification on twitter aspect-based sentiment analysis. Appl. Intell. 48(5):1218–1232. https://doi.org/10.1007/s10489-017-1098-6

    Article  Google Scholar 

  7. Marcheggiani D, Täckström O, Esuli A, Sebastiani F (2014) Hierarchical multi-label conditional random fields for aspect-oriented opinion mining. In: de Rijke M, Kenter T, de Vries AP, Zhai C, de Jong F, Radinsky K, Hofmann K (eds) Advances in Information Retrieval. Springer, Cham, pp 273–285

    Chapter  Google Scholar 

  8. Mikolov T, Zweig G(2012) Context dependent recurrent neural network language model. In: 2012 IEEE Spoken Language Technology Workshop (SLT), pp. 234– 239 https://doi.org/10.1109/SLT.2012.6424228

  9. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput. 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  10. Gers FA, Schmidhuber J, Cummins FA (2000) Learning to forget: Continual prediction with LSTM. Neural Comput. 12(10):2451–2471. https://doi.org/10.1162/089976600300015015

    Article  Google Scholar 

  11. Cho K, van Merriënboer B, Gulcehre C, Bahdanau D, Bougares F, Schwenk H, Bengio Y(2014) Learning phrase representations using RNN encoder–decoder for statistical machine translation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1724– 1734. Association for Computational Linguistics, Doha, Qatar

  12. Kim Y(2014) Convolutional neural networks for sentence classification. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1746– 1751. Association for Computational Linguistics, Doha, Qatar

  13. De Luca P, Galletti A, Giunta G, Marcellino L (2020) Accelerated gaussian convolution in a data assimilation scenario. In: Krzhizhanovskaya VV, Závodszky G, Lees MH, Dongarra JJ, Sloot PMA, Brissos S, Teixeira J (eds) Computational Science - ICCS 2020. Springer, Cham, pp 199–211

  14. Bo D, Wang X, Shi C, Shen H(2021) Beyond low-frequency information in graph convolutional networks. CoRR abs/2101.00797

  15. Zhang C, Li Q, Song D(2019) Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4568– 4578. Association for Computational Linguistics, Hong Kong, China

  16. Akhtar MS, Ekbal A, Cambria E (2020) How intense are you? Predicting intensities of emotions and sentiments using stacked ensemble [application notes]. IEEE Comput Intell Mag 15(1):64–75. https://doi.org/10.1109/MCI.2019.2954667

    Article  Google Scholar 

  17. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) NRC-Canada-2014: detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp. 437– 442. Association for Computational Linguistics, Dublin, Ireland

  18. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp 2514– 2523. Association for Computational Linguistics, Melbourne, Australia

  19. Ruder S, Ghaffari P, Breslin JG (2016) A Hierarchical Model of Reviews for Aspect-Based Sentiment Analysis. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 999– 1005. Association for Computational Linguistics, Austin, Texas

  20. Zhang M, Zhang Y, Vo D (2016) Gated Neural Networks for Targeted Sentiment Analysis. In: Schuurmans D, Wellman MP (Eds.) Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, pp 3087– 3093

  21. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606– 615. Association for Computational Linguistics, Austin, Texas

  22. Tang D, Qin B, Feng X, Liu T (2016) Effective LSTMs for Target-Dependent Sentiment Classification. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: technical papers, pp 3298– 3307. The COLING 2016 Organizing Committee, Osaka, Japan

  23. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Sierra C (Ed.) Proceedings of the twenty-sixth international joint conference on artificial intelligence, pp 4068– 4074

  24. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 3433– 3442. Association for Computational Linguistics, Brussels, Belgium

  25. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent Attention Network on Memory for Aspect Sentiment Analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 452– 461. Association for Computational Linguistics, Copenhagen, Denmark

  26. Cai H, Tu Y, Zhou X, Yu J, Xia R (2020) Aspect-category based sentiment analysis with hierarchical graph convolutional network. In: Scott D, Bel N, Zong C (Eds.) Proceedings of the 28th international conference on computational linguistics, pp 833– 843

  27. Zhang M, Qian T (2020) Convolution over hierarchical syntactic and lexical graphs for aspect level sentiment analysis. In: Webber B, Cohn T, He Y, Liu Y (Eds.) Proceedings of the 2020 conference on empirical methods in natural language processing, pp 3540– 3549

  28. Pennington J, Socher R, Manning C (2014) GloVe: Global Vectors for Word Representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532– 1543. Association for Computational Linguistics, Doha, Qatar

  29. Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, volume 1 (Long and Short Papers), pp. 4171– 4186. Association for Computational Linguistics, Minneapolis, Minnesota

  30. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent Twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 2: Short Papers), pp. 49– 54. Association for Computational Linguistics, Baltimore, Maryland

  31. Wang S, Mazumder S, Liu B, Zhou M, Chang Y (2018) Target-sensitive memory networks for aspect sentiment classification. In: Proceedings of the 56th annual meeting of the association for computational linguistics (Volume 1: Long Papers), pp. 957– 967. Association for Computational Linguistics, Melbourne, Australia

  32. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 214– 224. Association for Computational Linguistics, Austin, Texas

  33. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. CoRR abs/1902.09314

  34. Gu S, Zhang L, Hou Y, Song Y (2018) A Position-Aware Bidirectional Attention Network for Aspect-Level Sentiment Analysis. In: Proceedings of the 27th international conference on computational linguistics, pp 774– 784. association for computational linguistics, Santa Fe, New Mexico, USA

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, China

    Xiaowen Li, Ran Lu & Peiyu Liu

  2. School of Information Science and Electrical Engineering, Shandong Jiaotong University, Jinan, 250357, China

    Zhenfang Zhu

Authors
  1. Xiaowen Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Ran Lu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Peiyu Liu
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Zhenfang Zhu
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Ran Lu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, X., Lu, R., Liu, P. et al. Graph convolutional networks with hierarchical multi-head attention for aspect-level sentiment classification. J Supercomput 78, 14846–14865 (2022). https://doi.org/10.1007/s11227-022-04480-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-022-04480-w

Keywords