Skip to main content
SpringerLink
Log in

A BERT-based multi-semantic learning model with aspect-aware enhancement for aspect polarity classification

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-Based Sentiment Classification (ABSA), predicting the sentimental tendency towards given aspects, is an important branch in natural language understanding. However, in the existing deep learning models for ABSA, there is a contradiction between the fine sentiment analysis and the small amount of corpus. To solve this contradiction, we propose a BERT-based Multi-Semantic Learning (BERT-MSL) model with aspect-aware enhancement for aspect polarity classification, which follows the Transformer structure in BERT and uses lightweight multi-head self-attentions for encoding. First, we make full use of the extensive pre-training and post-training of the BERT model to obtain the initialization parameters with rich knowledge for our BERT-MSL model, so that our model can be quickly adapted to the ABSA task only by fine-tuning on a small corpus. Second, to achieve the fine sentiment analysis centered on aspect target, we propose a BERT-based multi-semantic learning model composed of the left-side local semantic, right-side local semantic, aspect target semantic and global semantic learning modules, and propose an aspect-aware enhancement method based on BERT and multi-head attention. Third, we propose two alternative semantic merging methods to generate the final expressive-powerful sentiment semantics for ABSA. Furthermore, to expand the application scope of our model, we design an advanced structure for our model by introducing a CNN-based semantic refinement layer. Experimental results on five SemEval and Twitter datasets demonstrate that our model improves the stability and robustness of ABSA and significantly outperforms some of the state-of-the-art models under the BERT Post-Training (BERT-PT) environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. https://storage.googleapis.com/bert_models/2020_02_20/uncased_L-12_H-768_A-12.zip

  2. https://howardhsu.github.io/

  3. https://www.yelp.com/dataset/challenge

References

  1. Pang B, Lee L (2008) Opinion mining and sentiment analysis. Foundations and trends in information retrieval 2(1):1–135

  2. Liu B (2012) Sentiment analysis and opinion mining. Synthesis Lectures on Human Language Technologies 5(1):1–167

    Article  Google Scholar 

  3. Kanayama H, Nasukawa T (2006) Fully automatic lexicon expansion for domain-oriented sentiment analysis. In: Processdings of the 2006 conference on empirical methods in natural language processing. Association for Computational Linguistics, Sydney, Australia, 22–23 July 2006, pp 355–363

  4. Liao WX, Zeng B, Yin XW, Wei PF (2021) An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa. Appl Intell 51(6):3522–3533

    Article  Google Scholar 

  5. Tang D, Qin B, Feng X, Liu T (2016) Effective LSTMs for target-dependent sentiment classification. In: Proceedings of the COLING 2016, the 26th international conference on computational linguistics: technical papers, Osaka, Japan, 11-17 December 2016, pp 3298-3307

  6. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: Empirical methods in natural language processing, pp 214–224

  7. Li M, Chen L, Zhao J, Li Q (2021) Sentiment analysis of Chinese stock reviews based on BERT model. Appl Intell 51(7):5016–5024

    Article  Google Scholar 

  8. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606-615

  9. Ma D, Li S, Zhang X, Wang H (2017) Interactive attentions networks for aspect-level sentiment classification. In: Proceedings of the 26th international joint conference on artificial intelligence, Melbourne, Australia, 19-25 August 2017, pp 4068-4074

  10. Song Y, Wang J, Jiang T, Liu Z, Rao Y (2019) Attentional encoder network for targeted sentiment classification. In: arXiv preprint arXiv:1902.09314

  11. Xu Q, Zhu L, Dai T, Yan C (2020) Aspect-based sentiment classification with multi-attention network. Neurocomputing 388(3):135–143

    Article  Google Scholar 

  12. Xu L, Bing L, Lu W, Huang F (2020) Aspect sentiment classification with aspect-specific opinion spans. In: Proceedings of EMNLP 2020, pp 3561-3567

  13. Yang H, Zeng B, Yang J, Song Y, Xu R (2021) A multi-task learning model for Chinese-oriented aspect polarity classification and AspectTerm extraction. Neurocomputing 419:344–356

    Article  Google Scholar 

  14. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of EMNLP, pp 452-461

  15. Bahdanau D, Cho K, Bengio Y (2015) Neural machine translation by jointly learning to align and translate. In: Proceedings of ICLR

  16. Xu H, Liu B, Shu L, Yu PS (2019) BERT Post-Training for Review Reading Comprehension and Aspect-based Sentiment Analysis. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1, pp 2324–2335

  17. Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 conference of NAACL, pp 4171–4186

  18. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: 31st conference on neural information processing systems (NIPS 2017), pp 5998–6008

  19. Zhou J, Huang X, Hu Q, He L (2020) SK-GCN: modeling syntax and knowledge via graph convolutional network for aspect-level sentiment classification. Knowl-Based Syst 205(3):106292

    Article  Google Scholar 

  20. Song W, Wen Z, Xiao Z, Park S (2021) Semantics perception and refinement network for aspect-based sentiment analysis. Knowl-Based Syst 214:106755

    Article  Google Scholar 

  21. Chen Z, Qian T (2019) Transfer capsule network for aspect level sentiment classification. In: Proceedings of the 57th annual meeting of the Association for Computational Linguistics, pp 547–556

  22. Weston J, Chopra S, Bordes A (2014) Memory networks. In: arXiv preprint Xiv:1410.3916

  23. Wu C, Xiong Q, Yang Z, Gao M (2021) Residual attention and other aspects module for aspect-based sentiment analysis. Neurocomputing 435:42–52

    Article  Google Scholar 

  24. He R, Lee W, Ng H, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. In: arXiv preprint arXivL 1806.04346

  25. Peters ME, Neumann M, Lyyer M, Gardner M, Clark C, Lee K, Zettlemoyer L (2018) Deep contextualized word representations. In: NAACL, pp 2227–2237

  26. Radford A, Narasimhan K, Salimans T, Sutskever I (2018) Improving language understanding by generative pre-training. In: OpenAI Blog

  27. Radford A, Wu J, Child R, Luan D, Amodei D, Sutskever I (2019) Language models are unsupervised multitask learners. In: OpenAI Blog

  28. Rietzler A, Stabinger S, Opitz P, Engl S (2019) Domain adaptation through BERT language model finetuning for aspect-target sentiment classification. In: arXiv: 1908.11860

  29. Liu Y, Ott M, Goyal N, Du J, Joshi M, Chen D, Levy O, Lewis M, Zettlemoyer L, Stoyanov V (2019) Roberta: a robustly optimized bert pretraining approach. In: arXiv: 1907.11692v1

  30. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) Semeval-2014 task 4: aspect based sentiment analysis. In: Proceedings of the 8th international workshop on semantic evaluation(SemEval2014), Dublin, Irland,23-24 August 2014, pp 27-35

  31. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) SemEval-2015 task 12: aspect based sentiment analysis. In proceedings of SemEval

  32. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, AL-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O et al (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), San Diego, CA, USA, 16–17 June 2016, pp 19–30

  33. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recurrent neural network for target-dependent twitter sentiment classification. In: Proceedings of ACL, volume 2, pp 49–54

  34. Wang B, Lu W (2018) Learning latent opinions for aspect-level sentiment classification. In: Proceedings of AAAI

  35. He R, Lee WS, Ng TH, Dahlmeier D (2018) Effective attention modeling for aspect-level sentiment classification. In: Processdings of COLING, pp 1121–1131

  36. He R, McAuley J (2016) Ups and downs: modeling the visual evolution of fashion trends with one-class collaborative filtering. In: World Wide Web

  37. Rajpurkar R, Zhang J, Lopyrev K, Liang P (2016) SQuAD: 100,000+ questions for machine comprehension of text. In: arXiv preprint arXiv:1606.05250

  38. Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of ICLR, vol 5

  39. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: Proceedings of ACL, pp 946-956

Download references

Acknowledgements

This work was supported in part by the National Natural Science Foundation of China under the contract number 62062012, the Natural Science Foundation of Guangxi of China under the contract number 2020GXNSFAA159082, the National Natural Science Foundation of China under the contract number 61967003, and Guangxi Collaborative Innovation Center of Multi-source Information Integration and Intelligent Processing.

Author information

Authors and Affiliations

  1. School of Computer Science and Engineering, Guangxi Normal University, Guilin, 541004, China

    Xinhua Zhu, Yuxiang Zhu & Yishan Chen

  2. Faculty of Education, Guangxi Normal University, Guilin, 541004, China

    Lanfang Zhang

  3. Business College, Guilin Tourism University, Guilin, 541006, China

    Yishan Chen

Authors
  1. Xinhua Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yuxiang Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lanfang Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yishan Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yishan Chen.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhu, X., Zhu, Y., Zhang, L. et al. A BERT-based multi-semantic learning model with aspect-aware enhancement for aspect polarity classification. Appl Intell 53, 4609–4623 (2023). https://doi.org/10.1007/s10489-022-03702-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03702-1

Keywords

Search

Navigation