Skip to main content
SpringerLink
Log in

PG-RNN: using position-gated recurrent neural networks for aspect-based sentiment classification

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Recently, recurrent neural networks (RNN) have achieved great success in the aspect-based sentiment classification task. Existing approaches always focus on capture the local (attentive) representation or global representation independently, while how to integrate them is not well studied. To address this problem, we propose a Position-Gated Recurrent Neural Networks (PG-RNN) model that considered aspect word position information. PG-RNN can integrate global and local information dynamically for aspect-based sentiment classification. Specifically, first, we propose a positional RNN model to integrate the aspect position information into the sentence encoder to enhance the latent representation. Unlike the existing work, we use kernel function to model position information instead of discrete distance values. Second, we design a representation absorption gating to absorb local positional representation and global representation dynamically. Experiments on five benchmark datasets show the significant advantages of our proposed model. More specifically, we achieve a maximum improvement of 7.38% over the classic attention-based RNN model in terms of accuracy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. Particularly, more kernel functions can be used to measure the distances though we only evaluate on two classic ones.

  2. http://alt.qcri.org/semeval2014/.

  3. http://alt.qcri.org/semeval2015/task12/.

  4. http://alt.qcri.org/semeval2016/task5/.

References

  1. Liu Bing (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167

    Article  Google Scholar 

  2. Zhou Jie , Tian Junfeng , Wang Rui, Wu Yuanbin, Xiao Wenming, He Liang (2020). Sentix: A sentiment-aware pre-trained model for cross-domain sentiment analysis. In: Proceedings of the 28th International Conference on Computational Linguistics, pp. 568–579

  3. Jie Zhou, Yuanbin Wu, Changzhi Sun, and Liang He. (2021) Is “hot pizza” positive or negative? mining target-aware sentiment lexicons. In: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 608–618

  4. Cambria Erik, Poria Soujanya, Bajpai Rajiv, Schuller Björn (2016) Senticnet 4: a semantic resource for sentiment analysis based on conceptual primitives. COLING, Technical Papers, In, pp 2666–2677

    Google Scholar 

  5. J Cheng, S Zhao, J Zhang, I King, X Zhang, and H Wang. (2017) Aspect-level sentiment classification with heat (hierarchical attention) network. In: CIKM, pp. 97–106

  6. M Pontiki, D Galanis, H Papageorgiou, I Androutsopoulos, S Manandhar, M AL-Smadi, M Al-Ayyoub, Y Zhao, B Qin, O De Clercq, et al. (2016) Semeval-2016 task 5: aspect based sentiment analysis. In: SemEval-2016, pp. 19–30

  7. Schouten K, Frasincar F (2016) Survey on aspect-level sentiment analysis. TKDE 1(1):1

    Google Scholar 

  8. Jie Z, Qin C, Xiangji HJ, Vivian HQ, Liang H (2020) Position-aware hierarchical transfer model for aspect-level sentiment classification. Inf Sci 513:1–16

    Article  Google Scholar 

  9. Zhou J, Jimmy XH, Qinmin VH, He L (2020) Sk-gcn: modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification. Knowl-Based Syst 205:106292

    Article  Google Scholar 

  10. Tang D, Q Bing, F Xiaocheng, L Ting (2016). Effective LSTMs for target-dependent sentiment classification. In: COLING, pp. 3298–3307

  11. C dos Santos, M Tan, B Xiang, and B Zhou. (2016) Attentive pooling networks. arXiv preprint arXiv:1602.03609

  12. Xu L, Liu J, Wang L, Yin C (2017). Aspect based sentiment analysis for online reviews. In: Advances in Computer Science and Ubiquitous Computing, pp. 475–480

  13. H Zhao, Z Lu, and P Poupart. (2015) Self-adaptive hierarchical sentence model. In: IJCAI, pp. 4069–4076

  14. Zhou C, Sun C, Liu Z, Lau F (2015). A C-LSTM neural network for text classification. arXiv preprint arXiv:1511.08630

  15. J Liu, Y Zhang (2017) Attention modeling for targeted sentiment. In: ACL: Volume 2, Short Papers, pp. 572–577

  16. Ma D, Li S, Zhang X, Wang H (2017). Interactive attention networks for aspect-level sentiment classification. In: IJCAI, pp. 4068–4074

  17. Y Wang, M Huang, L Zhao, et al. (2016) Attention-based LSTM for aspect-level sentiment classification. In: ACL, pp. 606–615

  18. Yang M, Tu W, Wang J, Xu F, Chen X (2017). Attention based LSTM for target dependent sentiment classification. In: AAAI, pp. 5013–5014

  19. P Chen, Z Sun, L Bing, and W Yang. (2017) Recurrent attention network on memory for aspect sentiment analysis. In: EMNLP, pp. 452–461

  20. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: ACL

  21. X Li and W Lam. (2017) Deep multi-task learning for aspect term extraction with memory interaction. In: EMNLP, pp. 2886–2892

  22. Tang D, Qin B, Liu T (2016) . Aspect level sentiment classification with deep memory network. In: ACL, pp. 214–224

  23. A Vaswani, N Shazeer, N Parmar, J Uszkoreit, L Jones, AN Gomez, L Kaiser, and I Polosukhin. (2017) Attention is all you need. In: Advances in neural information processing systems, pp. 5998–6008

  24. L Dong, F Wei, C Tan, D Tang, M Zhou, and K Xu (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: ACL: Volume 2, Short Papers, vol 2, pp. 49–54

  25. Jiang L, Mo Y, Zhou M, Liu X, Zhao T (2011) Target-dependent twitter sentiment classification. ACL: HLT 1:151–160

    Google Scholar 

  26. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Detecting aspects and sentiment in customer reviews. SemEval 2014:437–442

    Google Scholar 

  27. Nguyen TH, Shirai K (2015). Phrasernn: phrase recursive neural network for aspect-based sentiment analysis. In: EMNLP, pp. 2509–2514

  28. Vo DT, Zhang Y (2015). Target-dependent twitter sentiment classification with rich automatic features. In: IJCAI, pp. 1347–1353

  29. Jianxing Y, Zha ZJ, Wang M, Chua TS (2011) Aspect ranking: identifying important product aspects from online consumer reviews. ACL: HLT-Volume 1:1496–1505

  30. Zhang M, Zhang Y, Vo DT (2016). Gated neural networks for targeted sentiment analysis. In: AAAI, pp. 3087–3093

  31. Pang Bo, Lee Lillian, Vaithyanathan Shivakumar (2002) Thumbs up?: sentiment classification using machine learning techniques. ACL 10:79–86

    Google Scholar 

  32. He R, WS Lee, HT Ng, and D Dahlmeier (2018) Exploiting document knowledge for aspect-level sentiment classification, In: ACL

  33. J Tang, Z Lu, J Su, Y Ge, L Song, L Sun, and J Luo. (2019) Progressive self-supervised attention learning for aspect-level sentiment analysis. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 557–566

  34. Hochreiter Sepp, Schmidhuber Jürgen (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  35. Bahdanau D, Cho K, Bengio Y (2014). Neural machine translation by jointly learning to align and translate. In: CoRR, abs/1409.0473

  36. Y Ma, H Peng, and E Cambria. (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm. In: AAAI

  37. Zhang C, Li Q, Song D (2019). Aspect-based sentiment classification with aspect-specific graph convolutional networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 4560–4570

  38. K Sun, R Zhang, S Mensah, Y Mao, and X Liu (2019) Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp. 5683–5692

  39. Wang S, Lo D, Xing Z, Jiang L (2011). Concern localization using information retrieval: an empirical study on linux kernel. In: 2011 18th Working Conference on Reverse Engineering, IEEE. pp. 92–96

  40. Moschitti Alessandro, Quarteroni Silvia (2011) Linguistic kernels for answer re-ranking in question answering systems. Inf Process Manag 47(6):825–842

    Article  Google Scholar 

  41. D Wang and E Nyberg. (2015) A long short-term memory model for answer sentence selection in question answering. In: ACL: Volume 2, Short Papers, vol 2, pp. 707–712

  42. Luong MT, Pham H, Manning CD (2015). Effective approaches to attention-based neural machine translation. In: Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 1412–1421

  43. Zhao J, Huang JX, Ye Z (2014) Modeling term associations for probabilistic information retrieval. TOIS 32(2):7

    Article  Google Scholar 

  44. Jozefowicz R, Zaremba W, Sutskever I (2015). An empirical exploration of recurrent network architectures. In: ICML, pp. 2342–2350

  45. L Dong, F Wei, C Tan, D Tang, M Zhou, and K Xu. (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, ACL 2014, June 22-27, 2014, Baltimore, MD, USA, Volume 2: Short Papers, pp. 49–54

  46. Chung J , Gulcehre C, Cho K, Bengio Y (2014). Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555

  47. J Pennington, R Socher, and C Manning (2014) Glove: global vectors for word representation. In: EMNLP, pp. 1532–1543

  48. Tieleman T, Hinton G (2012) Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural Netw Mach Learn 4(2):26–31

    Google Scholar 

  49. J Pennington, R Socher, and CD Manning (2014) Glove: global vectors for word representation. In Alessandro Moschitti, Bo Pang, and Walter Daelemans, editors, Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pp. 1532–1543

  50. Li Z, Ying Wei Y, Xiang ZZ, Li X (2019) Exploiting coarse-to-fine task transfer for aspect-level sentiment classification. Proceed AAAI Conf Artif Intell 33:4253–4260

    Google Scholar 

Download references

Acknowledgements

We greatly appreciate anonymous reviewers and the associate editor for their valuable and high quality comments that greatly helped to improve the quality of this article. This research is funded by the Science and Technology Commission of Shanghai Municipality (19511120200).

Author information

Authors and Affiliations

  1. Shanghai Engineering Research Center of Open Distance Education, Shanghai Open University, Shanghai, China

    Qingchun Bai

  2. School of Computer Science, Fudan University, Shanghai, China

    Jie Zhou

  3. School of Computer Science and Technology, East China Normal University, Shanghai, China

    Liang He

Authors
  1. Qingchun Bai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jie Zhou
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Liang He
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jie Zhou.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bai, Q., Zhou, J. & He, L. PG-RNN: using position-gated recurrent neural networks for aspect-based sentiment classification. J Supercomput 78, 4073–4094 (2022). https://doi.org/10.1007/s11227-021-04019-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-04019-5

Keywords

Search

Navigation