Skip to main content
SpringerLink
Log in

Aspect-Based Sentiment Analysis Using Adversarial BERT with Capsule Networks

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis aims to predict the sentiment polarity of a given aspect in a sentence. The previous methods based on RNNs and attention mechanisms mainly have two problems: (1) The datasets for the aspect-based sentiment analysis task are small. Thus, the superiority of the neural network is not fully utilized. (2) The existing studies use brute force to accurately locate the one-to-one correspondence between the target words and sentiment words, and they also lack the ability to identify deep semantic relationships. To address these issues, we propose adversarial BERT with capsule networks. Specifically, the pre-trained BERT model with an adversarial training mechanism is introduced for semantic representation to solve the issue of insufficiently mining sentence semantic information due to an excessively small amount of training data. In addition, we apply the characteristics of the tensor neuron and dynamic routing mechanism of the capsule network to further explore the in-depth information of the sentence, which is conducive to accurately determining out the logical relationship between the target words and the sentiment words. Moreover, we deploy label smoothing regularization to reduce overfitting by preventing a network from assigning the full probability to each training example during training. As far as we know, this paper is the first innovative attempt to solve the challenge of capturing aspect words and their corresponding modified content bound to small sample data learning in aspect-based sentiment analysis using the adversarial training mechanism of the capsule network. Through experiments on three benchmark datasets, and compared with the state-of-the-art baselines, the extensive results show that our model achieves competitive improvements of up to 1.01%, 0.22%, and 1.68%, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Hutto C, Gilbert E (2014) Vader: A parsimonious rule-based model for sentiment analysis of social media text. In: Proceedings of the international AAAI conference on web and social media, vol 8

  2. Mingfeng P, Yi G (2019) Attention based gru network for domain adaptation in sentiment classification. DEStech Transactions on Computer Science and Engineering (iccis)

  3. Xia R, Ding Z (2019) Emotion-cause pair extraction: a new task to emotion analysis in texts. arXiv preprint arXiv:1906.01267

  4. Tang D, Qin B, Liu T (2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1422–1432

  5. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893

  6. Othman R, Belkaroui R, Faiz R (2017) Extracting product features for opinion mining using public conversations in twitter. Procedia Comput Sci 112:927–935

    Article  Google Scholar 

  7. Li B, Yang P, Zhao H, Zhang P, Liu Z (2023) Hierarchical sliding inference generator for question-driven abstractive answer summarization. ACM Trans Inf Syst 41(1):1–27. https://doi.org/10.1145/3511891

    Article  Google Scholar 

  8. Yi M, Yang P, Du M, Ma R (2022) Dmadrl: a distributed multi-agent deep reinforcement learning algorithm for cognitive offloading in dynamic mec networks. Neural Process Lett 1:1–33

    Google Scholar 

  9. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 437–442

  10. Wagner J, Arora P, Cortes S, Barman U, Bogdanova D, Foster J, Tounsi L (2014) Dcu: Aspect-based polarity classification for semeval task 4

  11. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. arXiv preprint arXiv:1805.07043

  12. Huang B, Carley KM (2019) Parameterized convolutional neural networks for aspect level sentiment classification. arXiv preprint arXiv:1909.06276

  13. Tang D, Qin B, Feng X, Liu T (2015) Effective lstms for target-dependent sentiment classification. arXiv preprint arXiv:1512.01100

  14. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615

  15. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. arXiv preprint arXiv:1710.09829

  16. Kim J, Jang S, Park E, Choi S (2020) Text classification using capsules. Neurocomputing 376:214–221

    Article  Google Scholar 

  17. Zhang C, Li Y, Du N, Fan W, Yu PS (2018) Joint slot filling and intent detection via capsule neural networks. arXiv preprint arXiv:1812.09471

  18. Pang B, Lee L, Vaithyanathan,S (2002) Thumbs up? sentiment classification using machine learning techniques. arXiv preprint arXiv:cs/0205070

  19. Kaji N, Kitsuregawa M (2007) Building lexicon for sentiment analysis from massive collection of html documents. In: Proceedings of the 2007 joint conference on empirical methods in natural language processing and computational natural language learning (EMNLP-CoNLL), pp 1075–1083

  20. Rao D, Ravichandran D (2009) Semi-supervised polarity lexicon induction. In: Proceedings of the 12th Conference of the European chapter of the ACL (EACL 2009), pp 675–682

  21. Vo D-T, Zhang Y (2015) Target-dependent twitter sentiment classification with rich automatic features. In: 24th International joint conference on artificial intelligence

  22. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473

  23. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  24. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. arXiv preprint arXiv:1605.08900

  25. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 452–461

  26. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. arXiv preprint arXiv:1709.00893

  27. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805

  28. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Targeted sentiment classification with attentional encoder network. In: International conference on artificial neural networks. Springer, pp 93–103

  29. Karimi A, Rossi L, Prati A (2021) Adversarial training for aspect-based sentiment analysis with bert. In: 2020 25th International conference on pattern recognition (ICPR). IEEE, pp 8797–8803

  30. Hendrycks D, Gimpel K (2016) Bridging nonlinearities and stochastic regularizers with gaussian error linear units

  31. Madry A, Makelov A, Schmidt L, Tsipras D, Vladu A (2017) Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv:1706.06083

  32. Miyato T, Dai AM, Goodfellow I (2016) Adversarial training methods for semi-supervised text classification. arXiv preprint arXiv:1605.07725

  33. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014). Association for Computational Linguistics, Dublin, Ireland, pp 27–35. https://doi.org/10.3115/v1/S14-2004. https://aclanthology.org/S14-2004

  34. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short Papers), pp 49–54

  35. Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980

  36. Liu J, Zhang Y (2017) Attention modeling for targeted sentiment. In: Proceedings of the 15th conference of the European chapter of the association for computational linguistics: volume 2, short papers, pp 572–577

  37. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. arXiv preprint arXiv:1805.07043

  38. Wu Z, Li Y, Liao J, Li D, Li X, Wang S (2020) Aspect-context interactive attention representation for aspect-level sentiment classification. IEEE Access 8:29238–29248

    Article  Google Scholar 

  39. Wu C, Xiong Q, Gao M, Li Q, Yu Y, Wang K (2021) A relative position attention network for aspect-based sentiment analysis. Knowl Inf Syst 63(2):333–347

    Article  Google Scholar 

  40. Sun K, Zhang R, Mensah S, Mao Y, Liu X (2019) Aspect-level sentiment analysis via convolution over dependency tree. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 5679–5688

  41. Shuang K, Gu M, Li R, Loo J, Su S (2021) Interactive pos-aware network for aspect-level sentiment classification. Neurocomputing 420:181–196

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under Grant 62272100, the Consulting Project of Chinese Academy of Engineering under Grant 2023-XY-09, and in part by the Fundamental Research Funds for the Central Universities and the Academy-Locality Cooperation Project of Chinese Academy of Engineering under Grant JS2021ZT05.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Southeast University, Nanjing, China

    Peng Yang, Bing Li, Shunhang Ji & Meng Yi

  2. School of Cyber Science and Engineering, Southeast University, Nanjing, China

    Peng Yang & Penghui Zhang

  3. Key Laboratory of Computer Network and Information Integration (Southeast University), Ministry of Education, Nanjing, China

    Peng Yang, Penghui Zhang, Bing Li, Shunhang Ji & Meng Yi

Authors
  1. Peng Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Penghui Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Bing Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Shunhang Ji
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Meng Yi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Peng Yang.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethical Approval

All authors read and approved the final version of the manuscript.

Consent to Participate

All authors contributed to this work.

Consent for Publication

All authors have checked the manuscript and have agreed to the submission.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yang, P., Zhang, P., Li, B. et al. Aspect-Based Sentiment Analysis Using Adversarial BERT with Capsule Networks. Neural Process Lett 55, 8041–8058 (2023). https://doi.org/10.1007/s11063-023-11296-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-023-11296-z

Keywords

Search

Navigation