Skip to main content
SpringerLink
Log in

Multi-head self-attention based gated graph convolutional networks for aspect-based sentiment classification

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Aspect-based sentiment classification aims to predict the sentiment polarity of specific aspects appeared in a sentence. Nowadays, most current methods mainly focus on the semantic information by exploiting traditional attention mechanisms combined with recurrent neural networks to capture the interaction between the contexts and the targets. However, these models did not consider the importance of the relevant syntactical constraints. In this paper, we propose to employ a novel gated graph convolutional networks on the dependency tree to encode syntactical information, and we design a Syntax-aware Context Dynamic Weighted layer to guide our model to pay more attention to the local syntax-aware context. Moreover, Multi-head Attention is utilized for capturing both semantic information and interactive information between semantics and syntax. We conducted experiments on five datasets and the results demonstrate the effectiveness of the proposed model.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  1. Bo P, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Ret 2:1–135

    Article  Google Scholar 

  2. Bo P, Lee L et al (2008) Opinion mining and sentiment analysis. Found Trends Inf Ret 2(1–2):1–135

    Article  Google Scholar 

  3. Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: Proceedings of the 2nd international conference on learning representations

  4. Chen Z, Qian T (2019) Transfer capsule network for aspect level sentiment classification. In: Proceedings of ACL-2019, pp 547–556

  5. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa PP (2011) Natural language processing (almost) from scratch. J Mach Learn Res 12:2493–2537

    MATH  Google Scholar 

  6. Conneau A, Schwenk H, Barrault L, LeCun Y (2016) Very deep convolutional networks for text classification. In: EACL, pp 1107–1116

  7. Devlin J, Chang M, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of the north american chapter of the association for computational linguistics: human language technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pp 4171–4186

  8. Dong X, Shen J, Wang W et al (2018) Hyperparameter optimization for tracking with continuous deep q-learning[C]. In: Computer Vision and Pattern Recognition, pp 518–527

  9. Dong X, Shen J, Wang W et al (2019) Dynamical hyperparameter optimization via deep reinforcement learning in tracking[J]. IEEE Trans Pattern Anal Mach Intell

  10. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics, vol 2, pp 49–54

  11. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (Volume 2: Short Papers), Baltimore, MD, USA, 23–25, pp 49–54

  12. He R, Lee WS, Ng HT, Dahlmeier D (2018) Effective attention modeling for aspect-level sentiment classification. In: Proceedings of the 27th international conference on computational linguistics, pp 1121–1131

  13. Hu J, Shen J, Yang B et al (2020) Infinitely wide graph convolutional networks: semi-supervised learning via gaussian processes[J]. arXiv:Learning

  14. Huang B, Carley K (2018) Parameterized convolutional neural networks for aspect level sentiment classification. In: Proceedings of the 2018 conference on empirical methods in natural language processing, pp 1091–1096

  15. Huang B, Ou Y, Carley K (2018) Aspect level sentiment classification with attention-over-attention neural networks. In: International conference on social computing, behavioral-cultural modeling and prediction and behavior representation in modeling and simulation. Springer, pp 197– 206

  16. Kalchbrenner N, Grefenstette E, Blunsom P (2014) A convolutional neural network for modelling sentences. In: ACL, pp 655–665

  17. Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv:1609.02907

  18. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), Dublin, Ireland, pp 437–442

  19. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. arXiv:1805.01086

  20. Li L, Liu Y, Zhou AnQiao (2018) Hierarchical attention-based position-aware network for aspect-level sentiment analysis. In: Proceedings of the 22nd conference on computational natural language learning, pp 181–189

  21. Lin P, Yang M, Lai J (2019) Deep mask memory network with semantic dependency and context moment for aspect level sentiment classification. In: Proceedings of IJCAI-2019, pp 5088–5094

  22. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. In: Proceedings of the 26th international joint conference on artificial intelligence. AAAI Press, pp 4068–4074

  23. Marcheggiani D, Titov I (2017) Encoding sentences with graph convolutional networks for semantic role labeling. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 1506–1515

  24. Maria Pontiki DG, John Pavlopoulos HP, Ion Androutsopoulos SM (2014) Semeval-2014 task 4: SemEval-2014 Task 4: Aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), Dublin, Ireland, pp 27–35

  25. Nguyen TH, Shirai K (2015) Phrasernn: Phrase recursive neural network for aspect-based sentiment analysis. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 2509–2514

  26. Nicola DC, Aziz W, Titov I (2018) Question answering by reasoning across documents with graph convolutional networks arXiv 2019. arXiv:1808.09920

  27. Pennington J, Socher R, Manning C (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  28. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, de Clercq O et al (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), San Diego, CA, USA, pp 19–30

  29. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), Denver, Colorado, pp 486–495

  30. Qi S, Wang W, Jia B et al (2018) Learning human-object interactions by graph parsing neural networks[C]. In: European conference on computer vision, pp 407–423

  31. Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, MaxWelling (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference. Springer, pp 593–607

  32. Shen J, Tang X, Dong X et al (2019) Visual object tracking by hierarchical attention siamese network[J]. IEEE Trans Syst Man Cybern 1–13

  33. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. arXiv:1902.09314

  34. Tang D, Qin B, Feng X, Liu T (2016a) Effective lstms for target-dependent sentiment classification. In: Proceedings of COLING 2016, the 26th international conference on computational linguistics: Technical papers, pp 3298–3307

  35. Tang D, Qin B, Liu T (2016b) Aspect level sentiment classification with deep memory network. arXiv:1502.03167

  36. Wagner J et al (2014) DCU: Aspect-based polarity classification for semeval task 4. In: Proc. 8th Int. Workshop Semantic Eval. (SemEval), Dublin, Ireland, pp 223–229

  37. Wang Y, Huang M, Li Z et al (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615

  38. Wang W, Lu X, Shen J et al (2019) Zero-shot video object segmentation via attentive graph neural networks[C]. In: International conference on computer vision, pp 9236–9245

  39. Wang W, Shen J (2018) Deep visual attention prediction[J]. IEEE Trans Image Process 27(5):2368–2378

    Article  MathSciNet  Google Scholar 

  40. Wang W, Shen J, Ling H et al (2019) A deep network solution for attention and aesthetics aware photo cropping[J]. IEEE Trans Pattern Anal Mach Intell 41(7):1531–1544

    Article  Google Scholar 

  41. Wang J, Sun C, Li S, Liu X, Si L, Zhang M, Zhou G (2019a) Aspect sentiment classification towards question-answering with reinforced bidirectional attention network. In: Proceedings of ACL-2019, pp 3548–3557

  42. Wu D, Dong X, Shen J et al (2020) Reducing estimation bias via triplet-average deep deterministic policy gradient[J]. IEEE Trans Neural Netw 1–13

  43. Xiao L, Hu X, Chen Y, Xue Y, Gu D, Chen B, Zhang T (2020) Targeted sentiment classification based on attentional encoding and graph convolutional networks. Appl. Sci. 10:957

    Article  Google Scholar 

  44. Xu H, Liu B, Shu L, Yu PS (2019) Bert post-training for review reading comprehension and aspect-based sentiment analysis. arXiv:1904.02232

  45. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. arXiv:1502.03167

  46. You H, Tian S, Yu L et al (2020) Pixel-level remote sensing image recognition based on bidirectional word vectors[J]. IEEE Trans Geosci Remote Sens 58(2):1281–1293

    Article  Google Scholar 

  47. Zeng B, Yang H, Xu R, Zhou W, Han X (2019) LCF: A local context focus mechanism for aspect-based sentiment classification. Appl Sci 9:3389

    Article  Google Scholar 

  48. Zhang C, Li Q, Song D (2018) Aspect-based sentiment classification with aspect-specific graph convolutional networks. arXiv:1909.03477

  49. Zhang Y, Qi P, Manning C (2018) Graph convolution over pruned dependency trees improves relation extraction. arXiv:1809.10185

  50. Zhaoa P, Houb L, Wua O (2019) Modeling sentiment dependencies with graph convolutional networks for aspect-level sentiment classification. arXiv:1906.04501

Download references

Acknowledgements

This work was supported by China Scholarship Council, the National Statistical Science Research Project of China under Grant No. 2016LY98, the Science and Technology Department of Guangdong Province in China under Grant Nos. 2016A010101020, 2016A010101021 and 2016A010101022, the Characteristic Innovation Projects of Guangdong Colleges and Universities (Nos. 2018KTSCX049 and 2018GKTSCX069), the Science and Technology Plan Project of Guangzhou under Grant Nos. 201802010033 and 201903010013.

Author information

Authors and Affiliations

  1. Guangdong Provincial Key Laboratory of Quantum Engineering and Quantum Materials, GPETR Center for Quantum Precision Measurement, SPTE, School of Physics and Telecommunication Engineering, South China Normal University, Guangzhou, 510006, China

    Luwei Xiao, Xiaohui Hu, Yun Xue, Bingliang Chen, Donghong Gu & Bixia Tang

  2. School of Computing, Informatics, and Decision Systems Engineering, Arizona State University, Tempe, AZ, 85287, USA

    Yinong Chen

Authors
  1. Luwei Xiao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaohui Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yinong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yun Xue
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bingliang Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Donghong Gu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bixia Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Xiaohui Hu.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Xiao, L., Hu, X., Chen, Y. et al. Multi-head self-attention based gated graph convolutional networks for aspect-based sentiment classification. Multimed Tools Appl 81, 19051–19070 (2022). https://doi.org/10.1007/s11042-020-10107-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-10107-0

Keywords

Search

Navigation