Skip to main content
SpringerLink
Log in

POS-ATAEPE-BiLSTM: an aspect-based sentiment analysis algorithm considering part-of-speech embedding

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-based sentiment analysis (ABSA) is a granular sentiment classification task that involves identifying sentiment polarities toward aspects in a sentence. Performing ABSA on online e-commerce reviews is essential for understanding customers’ opinions about specific aspects of a product or service, which can help businesses make informed decisions to improve overall customer satisfaction. Although previous research on ABSA has demonstrated considerable achievements, ABSA remains challenging in Chinese. First, most previous approaches ignored the extraction of aspect multiword in Chinese ABSA tasks, leading to the omission of aspect words. Moreover, words with a specific part-of-speech (POS) in Chinese text may contain sentiment information. To effectively capture the information of sentiment words, POS information can be more extensively applied in the ABSA model. Herein, we propose a novel model named POS-ATAEPE-BiLSTM for ABSA in Chinese. The proposed model considers words and the corresponding POS information for training word vectors. In addition, the embedding layer contains text embedding, aspect embedding, and POS embedding. This study provides several key contributions. First, we propose a multiple-aspect-word extraction algorithm based on POS tagging and consider single aspect words and aspect multiwords for pre-training, reducing the omission of aspect words. Second, we introduce a novel framework for ABSA, which is based on POS-driven word embedding–training and contains POS information in the embedding layer to improve the performance of the ABSA model. We use three pre-training corpora (i.e., Chinese Wikipedia, online reviews, and news articles) for pre-training and demonstrate the robustness of our proposed model in Chinese ABSA tasks.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The datasets generated during the current study are available from the corresponding author upon reasonable request.

References

  1. Ahmed M, Chen Q, Li Z (2020) Constructing domain-dependent sentiment dictionary for sentiment analysis. Neural Comput Appl 32(18):14719–14732. https://doi.org/10.1007/s00521-020-04824-8

    Article  Google Scholar 

  2. Al-Smadi M, Al-Ayyoub M, Jararweh Y et al (2019) Enhancing Aspect-Based Sentiment Analysis of Arabic Hotels’ reviews using morphological, syntactic and semantic features. Inform Process Manag 56(2, SI):308–319. https://doi.org/10.1016/j.ipm.2018.01.006

    Article  Google Scholar 

  3. Basiri M, Nemati S, Abdar M et al (2021) ABCDM: An Attention-based Bidirectional CNN-RNN Deep Model for sentiment analysis. Future Gener Comp Sy 115:279–294. https://doi.org/10.1016/j.future.2020.08.005

    Article  Google Scholar 

  4. Bi J, Liu Y, Fan Z et al (2019) Modelling customer satisfaction from online reviews using ensemble neural network and effect-based Kano model. Int J Prod Res 57(22):7068–7088. https://doi.org/10.1080/00207543.2019.1574989

    Article  Google Scholar 

  5. Chatterjee A, Gupta U, Chinnakotla MK et al (2019) Understanding Emotions in Text Using Deep Learning and Big Data. Comput Hum Behav 93:309–317. https://doi.org/10.1016/j.chb.2018.12.029

    Article  Google Scholar 

  6. Devlin J, Chang M, Lee K, Toutanova K (2019) Bert: pre-training of deep bidirectional transformers for language understanding. In: North American chapter of the association for computational linguistics, pp 4171–4186

  7. Do H, Prasad P, Maag A et al (2019) Deep Learning for Aspect-Based Sentiment Analysis: A Comparative Review. Expert Syst Appl 118:272–299. https://doi.org/10.1016/j.eswa.2018.10.003

    Article  Google Scholar 

  8. Enríquez F, Troyano JA, López-Solaz T (2016) An approach to the use of word embeddings in an opinion classification task. Expert Syst Appl 66:1–6. https://doi.org/10.1016/j.eswa.2016.09.005

    Article  Google Scholar 

  9. Fu X, Wei Y, Xu F et al (2019) Semi-supervised Aspect-level Sentiment Classification Model based on Variational Autoencoder. Knowl-Based Syst 171:81–92. https://doi.org/10.1016/j.knosys.2019.02.008

    Article  Google Scholar 

  10. Gu T, Zhao H, Li M (2023) Effective inter-aspect words modeling for aspect-based sentiment analysis. Appl Intell 53(4):4366–4379. https://doi.org/10.1007/s10489-022-03630-0

    Article  Google Scholar 

  11. Hoang M, Bihorac OA, Rouces J (2019) Aspect-based sentiment analysis using Bert. In: Proceedings of the 22nd nordic conference on computational linguistics, pp 187–196

  12. Huang F, Li X, Yuan C et al (2022) Attention-Emotion-Enhanced Convolutional LSTM for Sentiment Analysis. IEEE T Neur Net Lear 33(9):4332–4345. https://doi.org/10.1109/TNNLS.2021.3056664

    Article  Google Scholar 

  13. Huang F, Yuan C, Bi Y et al (2022) Multi-granular document-level sentiment topic analysis for online reviews. Appl Intell 52(7):7723–7733. https://doi.org/10.1007/s10489-021-02817-1

    Article  Google Scholar 

  14. Huang F, Zhang S, Zhang J et al (2017) Multimodal learning for topic sentiment analysis in microblogging. Neurocomputing 253:144–153. https://doi.org/10.1016/j.neucom.2016.10.086

    Article  Google Scholar 

  15. Jung Y, Suh Y (2019) Mining the voice of employees: A text mining approach to identifying and analyzing job satisfaction factors from online employee reviews. Decis Support Syst 123:113074. https://doi.org/10.1016/j.dss.2019.113074

    Article  Google Scholar 

  16. Kontopoulos E, Berberidis C, Dergiades T et al (2013) Ontology-based sentiment analysis of twitter posts. Expert Syst Appl 40(10):4065–4074. https://doi.org/10.1016/j.eswa.2013.01.001

    Article  Google Scholar 

  17. Kumar JA, Trueman TE, Cambria E (2021) A Convolutional Stacked Bidirectional LSTM with a Multiplicative Attention Mechanism for Aspect Category and Sentiment Detection. Cogn Comput 13(6):1423–1432. https://doi.org/10.1007/s12559-021-09948-0

    Article  Google Scholar 

  18. Li X, Li T, Li S et al (2023) Learning fusion feature representation for garbage image classification model in human-robot interaction. Infrared Phys Techn 128:104457. https://doi.org/10.1016/j.infrared.2022.104457

    Article  Google Scholar 

  19. Liao W, Zeng B, Yin X et al (2021) An improved aspect-category sentiment analysismodel for text sentiment analysis based on RoBERTa. Appl Intell 51(6):3522–3533. https://doi.org/10.1007/s10489-020-01964-1

    Article  Google Scholar 

  20. Lin Y, Wang C, Song H et al (2021) Multi-Head Self-Attention Transformation Networks for Aspect-Based Sentiment Analysis. IEEE Access 9:8762–8770. https://doi.org/10.1109/ACCESS.2021.3049294

    Article  Google Scholar 

  21. Liu H, Fang S, Zhang Z et al (2022) MFDNet: Collaborative Poses Perception and Matrix Fisher Distribution for Head Pose Estimation. IEEE T Multimedia 24:2449–2460. https://doi.org/10.1109/TMM.2021.3081873

    Article  Google Scholar 

  22. Liu H, Liu T, Chen Y et al (2022) EHPE: Skeleton Cues-based Gaussian Coordinate Encoding for Efficient Human Pose Estimation. IEEE Trans Multimediahttps://doi.org/10.1109/TMM.2022.3197364

  23. Liu H, Liu T, Zhang Z et al (2022) ARHPE: Asymmetric Relation-Aware Representation Learning for Head Pose Estimation in Industrial Human-Computer Interaction. IEEE T Ind Inform 18(10):7107–7117. https://doi.org/10.1109/TII.2022.3143605

    Article  Google Scholar 

  24. Liu T, Wang J, Yang B et al (2021) Facial expression recognition method with multi-label distribution learning for non-verbal behavior understanding in the classroom. Infrared Phys Techn 112:103594. https://doi.org/10.1016/j.infrared.2020.103594

    Article  Google Scholar 

  25. Liu T, Wang J, Yang B et al (2021) NGDNet: Nonuniform Gaussian-label distribution learning for infrared head pose estimation and on-task behavior understanding in the classroom. Neurocomputing 436:210–220. https://doi.org/10.1016/j.neucom.2020.12.090

    Article  Google Scholar 

  26. Ma Y, Peng H, Khan T et al (2018) Sentic LSTM: a Hybrid Network for Targeted Aspect-Based Sentiment Analysis. Cogn Comput 10(4):639–650. https://doi.org/10.1007/s12559-018-9549-x

    Article  Google Scholar 

  27. Mikolov T, Sutskever I, Chen K, et al (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the conference on neural information processing systems, pp 3111–3119.

  28. Nazir A, Rao Y, Wu L et al (2022) Issues and Challenges of Aspect-based Sentiment Analysis: A Comprehensive Survey. IEEE T Affect Comput 13(2):845–863. https://doi.org/10.1109/TAFFC.2020.2970399

    Article  Google Scholar 

  29. Park H, Song M, Shin K (2020) Deep learning models and datasets for aspect term sentiment classification: Implementing holistic recurrent attention on target-dependent memories. Knowl-Based Syst 187:104825. https://doi.org/10.1016/j.knosys.2019.06.033

    Article  Google Scholar 

  30. Peng H, Ma Y, Li Y et al (2018) Learning multi-grained aspect target sequence for Chinese sentiment analysis. Knowl-Based Syst 148:167–176. https://doi.org/10.1016/j.knosys.2018.02.034

    Article  Google Scholar 

  31. Pennington J, Socher R, Manning CD (2014). Glove: Global vectors for word representation. In: Proceedings of the empirical methods in natural language processing. Vol. 14. Proceedings of the empirical methods in natural language processing, pp 1532–1543

  32. Shuang K, Ren X, Yang Q et al (2019) AELA-DLSTMs: Attention-Enabled and Location-Aware Double LSTMs for aspect-level sentiment classification. Neurocomputing 334:25–34. https://doi.org/10.1016/j.neucom.2018.11.084

    Article  Google Scholar 

  33. Song M, Park H, Shin K (2019) Attention-based long short-term memory network using sentiment lexicon embedding for aspect-level sentiment analysis in Korean. Inform Process Manag 56(3):637–653. https://doi.org/10.1016/j.ipm.2018.12.005

    Article  Google Scholar 

  34. Subrahmanian V, Reforgiato D (2008) AVA: Adjective-Verb-Adverb Combinations for Sentiment Analysis. IEEE Intell Syst 23(4):43–50

    Article  Google Scholar 

  35. Taboada M, Brooke J, Tofiloski M et al (2011) Lexicon-Based Methods for Sentiment Analysis. Comput Linguist 37(2):267–307

    Article  Google Scholar 

  36. Tang D, Qin B, Feng X, Liu T (2016) Effective LSTMs for target-dependent sentiment classification. In: EMNLP, pp 3298–3307

  37. Wang Y, Huang M, Zhao L et al (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the conference on empirical methods in natural language processing, pp 606–615

  38. Wang G, Zhang Z, Sun J et al (2015) POS-RS: A Random Subspace method for sentiment classification based on part-of-speech analysis. Inform Process Manag 51(4):458–479. https://doi.org/10.1016/j.ipm.2014.09.004

    Article  Google Scholar 

  39. Xu G, Zhang Z, Zhang T et al (2022) Aspect-level sentiment classification based on attention-BiLSTM model and transfer learning. Knowl-Based Syst 245:108586. https://doi.org/10.1016/j.knosys.2022.108586

    Article  Google Scholar 

  40. Xu Q, Zhu L, Dai T et al (2020) Aspect-based sentiment classification with multi-attention network. Neurocomputing 388:135–143. https://doi.org/10.1016/j.neucom.2020.01.024

    Article  Google Scholar 

  41. Yadav RK, Jiao L, Goodwin M et al (2021) Positionless aspect-based sentiment analysis using attention mechanism. Knowl-Based Syst 226:107136. https://doi.org/10.1016/j.knosys.2021.107136

    Article  Google Scholar 

  42. Zainuddin N, Selamat A, Ibrahim R (2018) Hybrid sentiment classification on twitter aspect-based sentiment analysis. Appl Intell 48(5, SI):1218–1232. https://doi.org/10.1007/s10489-017-1098-6

    Article  Google Scholar 

  43. Zeng J, Ma X, Zhou K (2019) Enhancing Attention-Based LSTM With Position Context for Aspect-Level Sentiment Classification. IEEE Access 7:20462–20471. https://doi.org/10.1109/ACCESS.2019.2893806

    Article  Google Scholar 

  44. Zhang J, Lu X, Liu D (2021) Deriving customer preferences for hotels based on aspect-level sentiment analysis of online reviews. Electron Commer R A 49:101094. https://doi.org/10.1016/j.elerap.2021.101094

    Article  Google Scholar 

  45. Zhang Y, Xu B, Zhao T (2020) Convolutional multi-head self-attention on memory foraspect sentiment classification. IEEE/CAA J Autom Sin 7(4):1038–1044. https://doi.org/10.1109/JAS.2020.1003243

    Article  Google Scholar 

  46. Zhang J, Zhang A, Liu D et al (2021) Customer preferences extraction for air purifiers based on fine-grained sentiment analysis of online reviews. Knowl-Based Syst 228:107259. https://doi.org/10.1016/j.knosys.2021.107259

    Article  Google Scholar 

  47. Zhao L, Liu Y, Zhang M et al (2021) Modeling label-wise syntax for fine-grained sentiment analysis of reviews via memory-based neural model. Inform Process Manag 58(5):102641. https://doi.org/10.1016/j.ipm.2021.102641

    Article  Google Scholar 

  48. Zhao M, Yang J, Zhang J et al (2022) Aggregated graph convolutional networks for aspect-based sentiment classification. Inform Sciences 600:73–93. https://doi.org/10.1016/j.ins.2022.03.082

    Article  Google Scholar 

  49. Zhao A, Yu Y (2021) Knowledge-enabled BERT for aspect-based sentiment analysis. Knowl-Based Syst 227:107220. https://doi.org/10.1016/j.knosys.2021.107220

    Article  Google Scholar 

  50. Zhou J, Chen Q, Huang JX et al (2020) Position-aware hierarchical transfer model for aspect-level sentiment classification. Inf Sci 513:1–16. https://doi.org/10.1016/j.ins.2019.11.048

    Article  Google Scholar 

  51. Zhou J, Huang JX, Hu QV et al (2020) Is position important? deep multi-task learning for aspect-based sentiment analysis. Appl Intell 50(10):3367–3378. https://doi.org/10.1007/s10489-020-01760-x

    Article  Google Scholar 

  52. Zhu X, Zhu Y, Zhang L et al (2023) A BERT-based multi-semantic learning model with aspect-aware enhancement for aspect polarity classification. Appl Intell 53(4):4609–4623. https://doi.org/10.1007/s10489-022-03702-1

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Guangdong University of Technology, Guangzhou, China

    Qizhi Zhao, Zan Mo & Mengting Fan

Authors
  1. Qizhi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zan Mo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengting Fan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Qizhi Zhao.

Ethics declarations

Conflict of interest

The authors have no relevant financial or non-financial interests to disclose.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhao, Q., Mo, Z. & Fan, M. POS-ATAEPE-BiLSTM: an aspect-based sentiment analysis algorithm considering part-of-speech embedding. Appl Intell 53, 27440–27458 (2023). https://doi.org/10.1007/s10489-023-04952-3

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-023-04952-3

Keywords

Search

Navigation