Skip to main content
SpringerLink
Log in

Make aspect-based sentiment classification go further: step into the long-document-level

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-based sentiment classification (ABSC) is a fine-grained analysis task that obtains different sentiment polarities contained in a single text from the views of different aspects. Its practicability draws so much attention from researchers that the number of related works grows explosively. However, existing works mainly aim to obtain polarities from short texts (shorter than 100 words), only a few works analyze documents (shorter than 500 words), but almost no work analyzes long documents (LD, longer than 500 words). This situation makes ABSC powerless when dealing with some texts like in-depth analysis articles. In this paper, we make ABSC step into the LD level by proposing the Hierarchical Aspect-Oriented Framework for Long Document (HAOFL). HAOFL solves two challenges that rarely appear in short texts and normal documents. The first is the too-long input sequence that can cause the model to forget previously learned information or ignore the tailed unlearned information. The second is the unstable sentiment information of the target aspect contained in LD, which increases the difficulty for a model to draw a proper result. HAOFL constructs the data transformation module, dependency processing module, and sentiment aggregation module to solve these two challenges. Numerical experiments prove HAOFL can solve the aforementioned challenges and achieve superior performance in an effective and resource-saving way. With HAOFL, the performances of popular ABSC models on LD are improved at most 8.69% of accuracy and 11.37% of F1-score. In terms of resource-consuming, up to 82.10% of training time and 71.03% of GPU memory are saved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. https://medium.com/@SultanAlQassemi/the-role-of-art-in-society-c7425fd76de6

  2. www.people.com.cn, www.cctv.com, news.ifeng.com, news.163.com, www.ce.cn, www.youth.cn, www.chinanews.com

  3. https://www.selenium.dev/

  4. https://github.com/Ghands/HAOFL

  5. https://pytorch.org/get-started/previous-versions/

  6. https://towardsdatascience.com/guide-to-classification-on-imbalanced-datasets-d6653aa5fa23

References

  1. Behdenna S, Barigou F, Belalem G (2018) Document level sentiment analysis: A survey. EAI Endorsed Trans Context-Aware Syst Appl 4(13)

  2. Pang Bo, Lee L (2008) Opinion mining and sentiment analysis foundations and trends in information retrieval, vol 2

  3. Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. In: Mining text data. Springer, pp 415–463

  4. Tang D, Qin B, Feng X, Liu T (2015) Effective lstms for target-dependent sentiment classification. arXiv:1512.01100

  5. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based lstm for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 606–615

  6. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification. arXiv:1709.00893

  7. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, pp 452–461

  8. Xue W, Li T (2018) Aspect based sentiment analysis with gated convolutional networks. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 2514–2523

  9. Wang Shuai, Mazumder S, Liu B, Zhou M, Chang Y (2018) Target-sensitive memory networks for aspect sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 957–967

  10. He R, Lee WS, Ng HT, Dahlmeier D (2019) An interactive multi-task learning network for end-to-end aspect-based sentiment analysis. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp 504–515

  11. Hu M, Zhao S, Zhang L, Cai K, Su Z, Cheng R, Shen X (2019) Can: Constrained attention networks for multi-aspect sentiment analysis. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 4593–4602

  12. Jiang L, Yu M, Zhou M, Liu X, Zhao T (2011) Target-dependent twitter sentiment classification. In: Proceedings of the 49th annual meeting of the association for computational linguistics: human language technologies, pp 151–160

  13. Kiritchenko S, Zhu X, Cherry C, Mohammad S (2014) Nrc-canada-2014: Detecting aspects and sentiment in customer reviews. In: Proceedings of the 8th international workshop on semantic evaluation (SemEval 2014), pp 437–442

  14. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  15. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv:1409.0473

  16. Ma Y, Peng H, Khan T, Cambria E, Hussain Amir (2018) Sentic lstm: a hybrid network for targeted aspect-based sentiment analysis. Cogn Comput 10(4):639–650

    Article  Google Scholar 

  17. Li X, Bing L, Lam W, Shi B (2018) Transformation networks for target-oriented sentiment classification. In: Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp 946–956

  18. Fan F, Feng Y, Zhao D (2018) Multi-grained attention network for aspect-level sentiment classification. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp 3433–3442

  19. Tang D, Qin B, Liu T (2016) Aspect level sentiment classification with deep memory network. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pp 214–224

  20. Zeng B, Yang H, Xu R, Zhou W, Han X (2019) Lcf: A local context focus mechanism for aspect-based sentiment classification. Applied Sciences 9(16):3389

    Article  Google Scholar 

  21. Peng H, Ma Y, Li Y, Cambria E (2018) Learning multi-grained aspect target sequence for chinese sentiment analysis. Knowl-Based Syst 148:167–176

    Article  Google Scholar 

  22. Ma Yukun, Peng Haiyun, Cambria Erik (2018) Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive lstm.. In: Aaai, pp 5876–5883

  23. Hussain A, Cambria E (2018) Semi-supervised learning for big social data analysis. Neurocomputing 275:1662–1673

    Article  Google Scholar 

  24. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) SemEval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics, Dublin, pp 27–35. https://www.aclweb.org/anthology/S14-2004

  25. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015), pp 486–495

  26. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O et al (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: 10th International Workshop on Semantic Evaluation (SemEval 2016)

  27. Dong L, Wei F, Tan C, Tang D, Zhou M, Xu K (2014) Adaptive recursive neural network for target-dependent twitter sentiment classification. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 2: Short papers), pp 49–54

  28. Li J, Yang H, Zong C (2018) Document-level multi-aspect sentiment classification by jointly modeling users, aspects, and overall ratings. In: Proceedings of the 27th International Conference on Computational Linguistics, pp 925–936

  29. Wang J, Sun C, Li S, Wang J, Si L, Zhang M, Liu X, Zhou G (2019) Human-like decision making: Document-level aspect sentiment classification via hierarchical reinforcement learning. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pp 5585–5594

  30. Zhang Q, Shi C (2019) An attentive memory network integrated with aspect dependency for document-level multi-aspect sentiment classification. In: Asian Conference on Machine Learning. PMLR, pp 425–440

  31. Ji Y, Liu H, He B, Xiao X, Wu H, Yu Y (2020) Diversified multiple instance learning for document-level multi-aspect sentiment classification. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp 7012–7023

  32. Yang J, Yang R, Lu H, Wang C, Xie J (2019) Multi-entity aspect-based sentiment analysis with context, entity, aspect memory and dependency information. ACM Trans Asian Low-Resour Lang Inf Process (TALLIP) 18(4):1–22

    Article  Google Scholar 

  33. Shi T, Rakesh V, Wang S, Reddy CK (2019) Document-level multi-aspect sentiment classification for online reviews of medical experts. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 2723–2731

  34. Joshi M, Chen D, Liu Y, Weld DS, Zettlemoyer L, Levy O (2020) Spanbert: Improving pre-training by representing and predicting spans. Trans Assoc Comput Linguist 8:64–77

    Article  Google Scholar 

  35. Turney PD (2002) Thumbs up or thumbs down? semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, pp 417–424

  36. Zhang Q, Wang B, Wu L, Huang X (2007) Fdu at trec 2007: Opinion retrieval of blog track. In: TREC, pp 500–274

  37. Tang D, Qin B, Liu T (2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of the 2015 conference on empirical methods in natural language processing, pp 1422–1432

  38. Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning. PMLR, pp 1188–1196

  39. Chen H, Sun M, Tu C, Lin Y, Liu Z (2016) Neural sentiment classification with user and product attention. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 1650–1659

  40. Zhou X, Wan X, Xiao J (2016) Attention-based lstm network for cross-lingual sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, pp 247–256

  41. Huang B, Ou Y, Carley KM (2018) Aspect level sentiment classification with attention-over-attention neural networks. In: International Conference on Social Computing, Behavioral-Cultural Modeling and Prediction and Behavior Representation in Modeling and Simulation. Springer, pp 197–206

  42. Devlin J, Chang Mx-W, Lee K, Toutanova K (2018) Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv:1810.04805

  43. Naseem U, Razzak I, Musial K, Imran M (2020) Transformer based deep intelligent contextual embedding for twitter sentiment analysis. Fut Gener Comput Syst 113:58–69

    Article  Google Scholar 

  44. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008

  45. Pennington J, Socher R, Manning CD (2014) Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pp 1532–1543

  46. Cohan A, Dernoncourt F, Kim DS, Bui T, Kim S, Chang W, Goharian N (2018) A discourse-aware attention model for abstractive summarization of long documents. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers), pp 615–621

  47. Wang Z, Ng P, Ma X, Nallapati R, Xiang B (2019) Multi-passage bert: A globally normalized bert model for open-domain question answering. arXiv:1908.08167

Download references

Author information

Authors and Affiliations

  1. Key Laboratory of High Confidence Software Technologies (Peking University), MoE, Beijing, China

    Zhenhao Wu, Jianbo Gao, Qingshan Li, Zhi Guan & Zhong Chen

  2. Department of Computer Science and Technology, EECS, Peking University, Beijing, China

    Zhenhao Wu, Jianbo Gao, Qingshan Li & Zhong Chen

  3. National Engineering Research Center for Software Engineering, Peking University, Beijing, China

    Zhi Guan

Authors
  1. Zhenhao Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jianbo Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qingshan Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhi Guan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zhong Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhenhao Wu.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Z., Gao, J., Li, Q. et al. Make aspect-based sentiment classification go further: step into the long-document-level. Appl Intell 52, 8428–8447 (2022). https://doi.org/10.1007/s10489-021-02836-y

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-021-02836-y

Keywords

Search

Navigation