Skip to main content
SpringerLink
Log in

Synchronously tracking entities and relations in a syntax-aware parallel architecture for aspect-opinion pair extraction

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Aspect-Opinion Pair Extraction (AOPE) task aims to capture each aspect with its corresponding opinions in user reviews. Entity recognition and relation detection are two fundamental subtasks of AOPE. Although recent works take interaction into account, the two subtasks are still relatively independent during calculation. Furthermore, since AOPE task has not been formally proposed for a long time, syntactic information does not attract much attention in the current deep learning models for AOPE. In this paper, we propose a model for Synchronously Tracking Entities and Relations (STER) to deal with AOPE. Specifically, we design a network consisting of a bank of gated RNNs, where we can track all entities of a review sentence in parallel. STER utilizes three features, i.e., context, syntax and relation, to learn the representation of each tracked entity and calculate the correlated degree between all entities synchronously at each time step. The entity representation and the correlated degree are highly dependent during calculation. Finally, they will be used for entity recognition and relation detection, respectively. Therefore, in STER, the two subtasks of AOPE can achieve sufficient interaction, which enhances their mutual heuristic effect heavily. To verify the effectiveness and adaptiveness of our model, we conduct experiments on two annotation versions of SemEval datasets. The results demonstrate that STER not only achieves advanced performances but adapts to different annotation strategies well.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

Availability of Data and Materials

All data and materials generated or analyzed during this study are included in this article.

Code Availability

The authors can provide their code to editors and reviewers during the review and publish the source code when this article becomes public.

Notes

  1. In our datasets, the length of a review sentence does not exceed 100 tokens. As for longer sentences, slicing should be adopted.

  2. We take cell states instead of hidden states as the final token representations, which can obtain better results.

References

  1. Pang B, Lee L (2007) Opinion mining and sentiment analysis. Found Trends Inf Retr 2(1-2):1–135. https://doi.org/10.1561/1500000011

    Google Scholar 

  2. Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. In: Mining Text Data. https://doi.org/10.1007/978-1-4614-3223-4_13. Springer, pp 415–463

  3. Liu B (2012) Sentiment analysis and opinion mining, Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. https://doi.org/10.2200/S00416ED1V01Y201204HLT016

  4. Kumar A V M, Nandkumar A N (2020) A survey on challenges and research opportunities in opinion mining. SN Comput Sci 1(3):171. https://doi.org/10.1007/s42979-020-00149-4

    Article  Google Scholar 

  5. Rana T A, Cheah Y-N (2016) Aspect extraction in sentiment analysis: comparative analysis and survey. Artif Intell Rev 46(4):459–483. https://doi.org/10.1007/s10462-016-9472-z

    Article  Google Scholar 

  6. Do H H, Prasad P W C, Maag A, Alsadoon A (2019) Deep learning for aspect-based sentiment analysis: A comparative review. Expert Syst Appl 118:272–299. https://doi.org/10.1016/j.eswa.2018.10.003

    Article  Google Scholar 

  7. Rana T A, Cheah Y-N (2017) A two-fold rule-based model for aspect extraction. Expert Syst Appl 89:273–285. https://doi.org/10.1016/j.eswa.2017.07.047

    Article  Google Scholar 

  8. Marcacini R M, Rossi R G, Matsuno I P, Rezende S O (2018) Cross-domain aspect extraction for sentiment analysis: A transductive learning approach. Decis Support Syst 114:70–80. https://doi.org/10.1016/j.dss.2018.08.009

    Article  Google Scholar 

  9. Luo H, Li T, Liu B, Wang B, Unger H (2019) Improving aspect term extraction with bidirectional dependency tree representation. IEEE ACM Trans Audio Speech Lang Process 27(7):1201–1212. https://doi.org/10.1109/TASLP.2019.2913094

    Article  Google Scholar 

  10. Chauhan G S, Meena Y K, Gopalani D, Nahta R (2020) A two-step hybrid unsupervised model with attention mechanism for aspect extraction. Expert Syst Appl 161:113673. https://doi.org/10.1016/j.eswa.2020.113673

    Article  Google Scholar 

  11. Fan Z, Wu Z, Dai X-Y, Huang S, Chen J (2019) Target-oriented opinion words extraction with target-fused neural sequence labeling. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Volume 1 (Long and Short Papers). https://doi.org/10.18653/v1/n19-1259, Minneapolis, pp 2509–2518

  12. Veyseh A P B, Nouri N, Dernoncourt F, Dou D, Nguyen T H (2020) Introducing syntactic structures into target opinion word extraction with deep learning. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online. https://doi.org/10.18653/v1/2020.emnlp-main.719, pp 8947–8956

  13. Zhang J, Li F, Zhang Z, Xu G, Wang Y, Wang X, Zhang Y (2021) Integrate syntax information for target-oriented opinion words extraction with target-specific graph convolutional network. Neurocomputing 440:321–335. https://doi.org/10.1016/j.neucom.2020.07.152

    Article  Google Scholar 

  14. Katiyar A, Cardie C (2016) Investigating lstms for joint extraction of opinion entities and relations. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, Volume 1: Long Papers. https://doi.org/10.18653/v1/p16-1087, Berlin, pp 919–929

  15. Li X, Lam W (2017) Deep multi-task learning for aspect term extraction with memory interaction. In: Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017. https://doi.org/10.18653/v1/d17-1310, Copenhagen, pp 2886–2892

  16. Yu J, Jiang J, Xia R (2019) Global inference for aspect and opinion terms co-extraction based on multi-task neural networks. IEEE ACM Trans Audio Speech Lang Process 27 (1):168–177. https://doi.org/10.1109/TASLP.2018.2875170

    Article  Google Scholar 

  17. Dai H, Song Y (2019) Neural aspect and opinion term extraction with mined rules as weak supervision. In: Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Volume 1: Long Papers. https://doi.org/10.18653/v1/p19-1520, Florence, pp 5268–5277

  18. Chen S, Liu J, Wang Y, Zhang W, Chi Z (2020) Synchronous double-channel recurrent network for aspect-opinion pair extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online. https://doi.org/10.18653/v1/2020.acl-main.582, pp 6515–6524

  19. Zhao H, Huang L, Zhang R, Lu Q, Xue H (2020) Spanmlt: A span-based multi-task learning framework for pair-wise aspect and opinion terms extraction. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online. https://doi.org/10.18653/v1/2020.acl-main.296, pp 3239–3248

  20. Wu Z, Ying C, Zhao F, Fan Z, Dai X, Xia R (2020) Grid tagging scheme for aspect-oriented fine-grained opinion extraction. In: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings. arXiv:2010.04640, pp 2576–2585

  21. Wu S, Fei H, Ren Y, Ji D, Li J (2021) Learn from syntax: Improving pair-wise aspect and opinion terms extraction with rich syntactic knowledge. In: Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI 2021. https://doi.org/10.24963/ijcai.2021/545, Virtual Event / Montreal, pp 3957–3963

  22. Wu S, Fei H, Ren Y, Li B, Li F, Ji D (2021) High-order pair-wise aspect and opinion terms extraction with edge-enhanced syntactic graph convolution. IEEE ACM Trans Audio Speech Lang Process 29:2396–2406. https://doi.org/10.1109/TASLP.2021.3095672

    Article  Google Scholar 

  23. Devlin J, Chang M-W, Lee K, Toutanova K (2019) BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Volume 1 (Long and Short Papers). https://doi.org/10.18653/v1/n19-1423, Minneapolis, pp 4171–4186

  24. Lafferty J D, McCallum A, Pereira F C N (2001) Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In: Proceedings of the Eighteenth International Conference on Machine Learning (ICML 2001). http://ssli.ee.washington.edu/courses/ee517sp07/papers/crf.pdf. Williams College, Williamstown, pp 282–289

  25. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the Tenth ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. https://doi.org/10.1145/1014052.1014073, Seattle, pp 168–177

  26. Popescu A-M, Etzioni O (2005) Extracting product features and opinions from reviews. In: HLT/EMNLP 2005, human language technology conference and conference on empirical methods in natural language processing, proceedings of the conference. https://www.aclweb.org/anthology/H05-1043/, Vancouver, pp 339–346

  27. Zhuang L, Jing F, Zhu X (2006) Movie review mining and summarization. In: Proceedings of the 2006 ACM CIKM International Conference on Information and Knowledge Management. https://doi.org/10.1145/1183614.1183625, Arlington, pp 43–50

  28. Qiu G, Liu B, Bu J, Chen C (2011) Opinion word expansion and target extraction through double propagation. Comput Linguist 37(1):9–27. https://doi.org/10.1162/coli_a_00034

    Article  Google Scholar 

  29. Jakob N, Gurevych I (2010) Extracting opinion targets in a single and cross-domain setting with conditional random fields. In: Proceedings of the 2010 conference on empirical methods in natural language processing, EMNLP 2010, A meeting of sigdat, a special interest group of the ACL. https://www.aclweb.org/anthology/D10-1101/. MIT Stata Center, Massachusetts, pp 1035–1045

  30. Li F, Han C, Huang M, Zhu X, Xia Y, Zhang S, Yu H (2010) Structure-aware review mining and summarization. In: COLING 2010, 23rd international conference on computational linguistics, proceedings of the conference. https://www.aclweb.org/anthology/C10-1074/, Beijing, pp 653–661

  31. Poria S, Cambria E, Gelbukh A F (2016) Aspect extraction for opinion mining with a deep convolutional neural network. Knowl Based Syst 108:42–49. https://doi.org/10.1016/j.knosys.2016.06.009

    Article  Google Scholar 

  32. Wang W, Pan S J, Dahlmeier D, Xiao X (2016) Recursive neural conditional random fields for aspect-based sentiment analysis. In: Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016. https://doi.org/10.18653/v1/d16-1059, Austin, pp 616–626

  33. Wang W, Pan S J, Dahlmeier D, Xiao X (2017) Coupled multi-layer attentions for co-extraction of aspect and opinion terms. In: Proceedings of the thirty-first AAAI conference on artificial intelligence. http://aaai.org/ocs/index.php/AAAI/AAAI17/paper/view/14441, Francisco, pp 3316–3322

  34. Chen Z, Qian T (2020) Relation-aware collaborative learning for unified aspect-based sentiment analysis. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online. https://doi.org/10.18653/v1/2020.acl-main.340, pp 3685–3694

  35. Gao L, Wang Y, Liu T, Wang J, Zhang L, Liao J (2021) Question-driven span labeling model for aspect-opinion pair extraction. In: Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Thirty-Third Conference on Innovative Applications of Artificial Intelligence, IAAI 2021, The Eleventh Symposium on Educational Advances in Artificial Intelligence, EAAI 2021. https://ojs.aaai.org/index.php/AAAI/article/view/17523, Virtual Event, pp 12875–12883

  36. Henaff M, Weston J, Szlam A, Bordes A, LeCun Y (2017) Tracking the world state with recurrent entity networks. In: 5th International Conference on Learning Representations, ICLR 2017, Conference Track Proceedings. https://openreview.net/forum?id=rJTKKKqeg, Toulon

  37. Liu F, Cohn T, Baldwin T (2018) Recurrent entity networks with delayed memory update for targeted aspect-based sentiment analysis. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT, Volume 2 (Short Papers). https://doi.org/10.18653/v1/n18-2045, New Orleans, pp 278–283

  38. Shen Y, Tan S, Sordoni A, Courville A C (2019) Ordered neurons: Integrating tree structures into recurrent neural networks. In: 7th International Conference on Learning Representations, ICLR 2019. https://openreview.net/forum?id=B1l6qiR5F7, New Orleans

  39. Pontiki M, Galanis D, Pavlopoulos J, Papageorgiou H, Androutsopoulos I, Manandhar S (2014) Semeval-2014 task 4: Aspect based sentiment analysis. In: Proceedings of the 8th International Workshop on Semantic Evaluation, SemEval@COLING 2014. https://doi.org/10.3115/v1/s14-2004, Dublin, pp 27–35

  40. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I (2015) Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2015. https://doi.org/10.18653/v1/s15-2082, Denver, pp 486–495

  41. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, Clercq O D, Hoste V, Apidianaki M, Tannier X, Loukachevitch N V, Kotelnikov E V, Bel N, Zafra S M J, Eryigit G (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation, SemEval@NAACL-HLT 2016. https://doi.org/10.18653/v1/s16-1002, San Diego, pp 19–30

  42. Zhang Y, Qi P, Manning C D (2018) Graph convolution over pruned dependency trees improves relation extraction. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. https://doi.org/10.18653/v1/d18-1244, Brussels, pp 2205–2215

Download references

Acknowledgements

This work is supported by the National Natural Science Foundation of China under grant No.61872163 and 61806084, Jilin Province Key Scientific and Technological Research and Development Project under grant No.20210201131GX, and Jilin Provincial Education Department project under grant No.JJKH20190160KJ.

Funding

This work is supported by the National Natural Science Foundation of China under grant No.61872163 and 61806084, Jilin Province Key Scientific and Technological Research and Development Project under grant No.20210201131GX, and Jilin Provincial Education Department project under grant No.JJKH20190160KJ.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Jilin University, Changchun, Jilin, 130012, China

    Yue Zhang, Tao Peng, Ridong Han & Lu Liu

  2. College of Software, Jilin University, Changchun, Jilin, 130012, China

    Tao Peng & Lu Liu

  3. Key Laboratory of Symbol Computation and Knowledge Engineer of the Ministry of Education, Changchun, Jilin, 130012, China

    Yue Zhang, Tao Peng, Ridong Han & Lu Liu

  4. Department of Linguistics, University of Washington, Seattle, WA, 98195, USA

    Jiayu Han

  5. School of Information Technology and Electrical Engineering, University of Queensland, Brisbane, QLD, 4072, Australia

    Lin Yue

Authors
  1. Yue Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tao Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ridong Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jiayu Han
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Lin Yue
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Lu Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Yue Zhang: Conceptualization, Methodology, Software, Visualization, Writing, Editing. Tao Peng: Methodology, Supervision, Funding acquisition, Reviewing, Validation. Ridong Han: Editing, Software, Visualization. Lin Yue: Editing, Reviewing. Jiayu Han: Editing, Reviewing. Lu Liu: Reviewing, Supervision, Validation, Funding acquisition.

Corresponding author

Correspondence to Tao Peng.

Ethics declarations

Ethics Approval

Not applicable.

Consent to Participate

Not applicable.

Consent for Publication

Not applicable.

Conflict of Interests

The authors have no conflicts of interest to declare that are relevant to the content of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, Y., Peng, T., Han, R. et al. Synchronously tracking entities and relations in a syntax-aware parallel architecture for aspect-opinion pair extraction. Appl Intell 52, 15210–15225 (2022). https://doi.org/10.1007/s10489-022-03286-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-03286-w

Keywords

Search

Navigation