Skip to main content
SpringerLink
Log in

A Convolutional Stacked Bidirectional LSTM with a Multiplicative Attention Mechanism for Aspect Category and Sentiment Detection

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

Traditionally, sentiment analysis is a binary classification task that aims to categorize a piece of text as positive or negative. This approach, however, can be too simplistic when the text under scrutiny contains more than one opinion target. Hence, aspect-based sentiment analysis provides fine-grained sentiment understanding of the product, service, or policy. Machine learning and deep learning algorithms play an important role in this kind of task. Also, attention mechanism has shown breakthrough in the field of natural language processing. Therefore, we propose a convolutional stacked bidirectional long short-term memory with a multiplicative attention mechanism for aspect category and sentiment polarity detection. More specifically, we treat the proposed model as a multiclass classification problem. The proposed model is evaluated using SemEval-2015 and SemEval-2016 dataset. Our proposed model outperforms state-of-the-art results in aspect-based sentiment analysis.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Cavallari S, Cambria E, Cai H, Chang K, Zheng V. Embedding both finite and infinite communities on graph. IEEE Comput Intell Mag. 2019;14(3):39–50.

    Article  Google Scholar 

  2. Camacho D, Panizo-LLedot A, Bello-Orgaz G, Gonzalez-Pardo A, Cambria E. The four dimensions of social network analysis: An overview of research methods, applications, and software tools. Information Fusion. 2020;63:88–120.

    Article  Google Scholar 

  3. Cambria E, Wang H, White B. Guest editorial: Big social data analysis. Knowl-Based Syst. 2014;69:1–2.

    Article  Google Scholar 

  4. Ragusa E, Cambria E, Zunino R, Gastaldo P. A survey on deep learning in image polarity detection: Balancing generalization performances and computational costs. Electronics. 2019;8(7):783.

  5. Stappen L, Baird A, Cambria E, Schuller B. Sentiment analysis and topic recognition in video transcriptions. IEEE Intell Syst. 2021;36(2):88–95.

    Article  Google Scholar 

  6. Lo SL, Cambria E, Chiong R, Cornforth D. Multilingual sentiment analysis: From formal to informal and scarce resource languages. Artif Intell Rev. 2017;48(4):499–527.

    Article  Google Scholar 

  7. Oueslati O, Cambria E, HajHmida MB, Ounelli H. A review of sentiment analysis research in arabic language. Futur Gener Comput Syst. 2020;112:408–30.

    Article  Google Scholar 

  8. Peng H, Cambria E, Hussain A. A review of sentiment analysis research in chinese language. Cogn Comput. 2017;9(4):423–35.

    Article  Google Scholar 

  9. Chaturvedi I, Cambria E, Vilares D. Lyapunov filtering of objectivity for Spanish sentiment model. In: IJCNN. 2016:4474-4481

  10. Li Y, Wang S, Ma Y, Pan Q, Cambria E. Popularity prediction on vacation rental websites. Neurocomputing. 2020;412:372–80.

    Article  Google Scholar 

  11. Li Y, Pan Q, Yang T, Wang S, Tang J, Cambria E. Learning word representations for sentiment analysis. Cogn Comput. 2017;9(6):843–51.

    Article  Google Scholar 

  12. Picasso A, Merello S, Ma Y, Oneto L, Cambria E. Technical analysis and sentiment embeddings for market trend prediction. Expert Syst Appl. 2019;135:60–70.

    Article  Google Scholar 

  13. Cambria E, Hussain A, Durrani T, Havasi C, Eckl C, Munro J. Sentic computing for patient centered applications. In: IEEE ICSP. 2010;1279–1282.

  14. Tolba M, Ouadfel S, Meshoul S. Hybrid ensemble approaches to online harassment detection in highly imbalanced data. Expert Syst Appl. 2021;175.

    Article  Google Scholar 

  15. Khatua A, Khatua A, Cambria E. Predicting political sentiments of voters from twitter in multi-party contexts. Appl Soft Comp. 2020;97(106743).

  16. Ma Y, Nguyen KL, Xing F, Cambria E. A survey on empathetic dialogue systems. Information Fusion. 2020;64:50–70.

    Article  Google Scholar 

  17. Xing F, Pallucchini F, Cambria E. Cognitive-inspired domain adaptation of sentiment lexicons. Inf Process Manag. 2019;56(3):554–64.

    Article  Google Scholar 

  18. Donadello I: OntoSenticNet 2: Enhancing reasoning within sentiment analysis. IEEE Intelligent Systems. 2021;36(5).

  19. Cambria E, Li Y, Xing F, Poria S, Kwok K. SenticNet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis. In: CIKM. 2020;105–114.

  20. Chaturvedi I, Ragusa E, Gastaldo P, Zunino R, Cambria E. Bayesian network based extreme learning machine for subjectivity detection. J Franklin Inst. 2018;355(4):1780–97.

    Article  MathSciNet  Google Scholar 

  21. Hussain A, Cambria E. Semi-supervised learning for big social data analysis. Neurocomputing. 2018;275:1662–73.

    Article  Google Scholar 

  22. Cambria E, Mazzocco T, Hussain A, Eckl C.:Sentic medoids: Organizing affective common sense knowledge in a multi-dimensional vector space. In: D.Liu, H.Zhang, M.Polycarpou, C.Alippi, H.He (eds.) Advances in Neural Networks, Lecture Notes in Computer Science, Springer-Verlag. 2011;6677:601-610

  23. Chaturvedi I, Ong YS, Tsang I, Welsch R, Cambria E. Learning word dependencies in text by means of a deep recurrent belief network. Knowl-Based Syst. 2016;108:144–54.

    Article  Google Scholar 

  24. Huang GB, Cambria E, Toh KA, Widrow B, Xu Z. New trends of learning in computational intelligence. IEEE Comput Intell Mag. 2015;10(2):16–7.

    Article  Google Scholar 

  25. Li Y, Pan Q, Wang S, Yang T, Cambria E. A generative model for category text generation. Inform Sci. 2018;450:301–15.

    Article  MathSciNet  Google Scholar 

  26. Zhao W, Peng H, Eger S, Cambria E, Yang M. Towards scalable and reliable capsule networks for challenging NLP applications. In: ACL. 2019;1549–1559.

  27. Susanto Y, Cambria E, Ng BC, Hussain A. Ten years of sentic computing. Cogn Comp. 2021;13.

  28. Cambria E, Poria S, Bisio F, Bajpai R, Chaturvedi I. The CLSA model: A novel framework for concept-level sentiment analysis. In: LNCS. Springer 2015;9042:3-22

  29. Satapathy R, Cambria E, Nanetti A, Hussain A. A review of shorthand systems: From brachygraphy to microtext and beyond. Cogn Comput. 2020;12(4):778–92.

    Article  Google Scholar 

  30. Chaturvedi I, Cambria E, Welsch R, Herrera F. Distinguishing between facts and opinions for sentiment analysis: Survey and challenges. Information Fusion. 2018;44:65–77.

    Article  Google Scholar 

  31. Sukthanker R, Poria S, Cambria E, Thirunavukarasu R. Anaphora and coreference resolution: A review. Information Fusion. 2020;59:139–62.

    Article  Google Scholar 

  32. Mehta Y, Majumder N, Gelbukh A, Cambria E. Recent trends in deep learning based personality detection. Artif Intell Rev. 2020;53:2313–39.

    Article  Google Scholar 

  33. Wang B, Liu M. Deep learning for aspect-based sentiment analysis. Stanford University report. 2015.

  34. Xue W, Li T. Aspect based sentiment analysis with gated convolutional networks. arXiv preprint https://arxiv.org/abs/1805.07043. 2018.

  35. Thet TT, Na JC, Khoo CS, Shakthikumar S. Sentiment analysis of movie reviews on discussion boards using a linguistic approach. In: Proceedings of the 1st international CIKM workshop on Topic-sentiment analysis for mass opinion. 2009;81–84.

  36. Pontiki M, Galanis D, Papageorgiou H, Manandhar S, Androutsopoulos I. Semeval-2015 task 12: Aspect based sentiment analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). 2015;486–495.

  37. Kim Y. Convolutional neural networks for sentence classification. arXiv preprint https://arxiv.org/abs/1408.5882. 2014

  38. Ruder S, Ghaffari P, Breslin JG. Insight-1 at semeval-2016 task 5: Deep learning for multilingual aspect-based sentiment analysis. arXiv preprint https://arxiv.org/abs/1609.02748. 2016.

  39. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.

    Article  Google Scholar 

  40. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: Encoder-decoder approaches. arXiv preprint https://arxiv.org/abs/1409.1259. 2014

  41. Chung J, Gulcehre C, Cho K, Bengio Y. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint https://arxiv.org/abs/1412.3555. 2014

  42. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint https://arxiv.org/abs/1409.0473. 2014.

  43. Hu M, Zhao S, Guo H, Cheng R, Su Z. Learning to detect opinion snippet for aspect-based sentiment analysis. arXiv preprint https://arxiv.org/abs/1909.11297. 2019.

  44. Ma Y, Peng H, Khan T, Cambria E, Hussain A. Sentic LSTM: a hybrid network for targeted aspect-based sentiment analysis. Cogn Comput. 2018;10(4):639–50.

    Article  Google Scholar 

  45. Luong MT, Pham H, Manning CD. Effective approaches to attention-based neural machine translation. arXiv preprint https://arxiv.org/abs/1508.04025. 2015.

  46. Tay Y, Tuan LA, Hui SC. Learning to attend via word-aspect associative fusion for aspect-based sentiment analysis. In: Thirty-second AAAI conference on artificial intelligence. 2018.

  47. Hazarika D, Poria S, Vij P, Krishnamurthy G, Cambria E, Zimmermann R. Modeling inter-aspect dependencies for aspect-based sentiment analysis. In: Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short Papers). 2018;266–270

  48. Tay Y, Tuan LA, Hui SC. Dyadic memory networks for aspect-based sentiment analysis. In: Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. 2017;107–116.

  49. Al-Smadi M, Talafha B, Al-Ayyoub M, Jararweh Y. Using long short-term memory deep neural networks for aspect-based sentiment analysis of arabic reviews. Int J Mach Learn Cybern. 2019;10(8):2163–75.

    Article  Google Scholar 

  50. Poria S, Chaturvedi I, Cambria E, Bisio F. Sentic LDA: Improving on LDA with semantic similarity for aspect-based sentiment analysis. In: 2016 international joint conference on neural networks (IJCNN). IEEE. 2016;4465–4473.

  51. Ruder S, Ghaffari P, Breslin JG. A hierarchical model of reviews for aspect-based sentiment analysis. arXiv preprint https://arxiv.org/abs/1609.02745. 2016.

  52. Pennington J, Socher R, Manning CD. Glove: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014;1532–1543.

  53. Repaka R, Pallelra RR, Koppula AR, Movva VS. Umduluth-cs8761-12: A novel machine learning approach for aspect based sentiment analysis. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). 2015;742–747.

  54. Kauer A, Moreira V. Ufrgs: Identifying categories and targets in customer reviews. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015). 2015;725–729.

  55. Hamdan H, Bellot P, Bechet F. Lsislif: Crf and logistic regression for opinion target extraction and sentiment polarity analysis. In: Proceedings of the 9th international workshop on semantic evaluation (SemEval 2015). 2015;753–758

  56. Saias J. Sentiue: Target and aspect based sentiment analysis in semeval-2015 task 12. Association for Computational Linguistics. 2015.

  57. Zhu P, Chen Z, Zheng H, Qian T. Aspect aware learning for aspect category sentiment analysis. ACM Transactions on Knowledge Discovery from Data (TKDD). 2019;13(6):1–21.

    Article  Google Scholar 

  58. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, DeClercq O, et al. Semeval-2016 task 5: Aspect based sentiment analysis. In: International workshop on semantic evaluation. 2016;19–30.

  59. Collobert R, Weston J, Bottou L, Karlen M, Kavukcuoglu K, Kuksa P. Natural language processing (almost) from scratch. J Mach Learn Res. 2011;12:2493–537.

    MATH  Google Scholar 

  60. Do BT. Aspect-based sentiment analysis using bitmask bidirectional long short term memory networks. In: The Thirty-First International Flairs Conference. 2018

  61. Choi E, Bahadori MT, Sun J, Kulas J, Schuetz A, Stewart W. Retain: An interpretable predictive model for healthcare using reverse time attention mechanism. In: Advances in Neural Information Processing Systems. 2016;3504–3512.

  62. Chollet, F., etal.: Keras. https://pypi.org/project/keras-self-attention/ (2015)

  63. Iqbal Z, Luo D, Henry P, Kazemifar S, Rozario T, Yan Y, Westover K, Lu W, Nguyen D, Long T, et al. Accurate real time localization tracking in a clinical environment using bluetooth low energy and deep learning. PLoS ONE. 2018;13(10).

    Article  Google Scholar 

  64. Hinton G, Srivastava N, Swersky K. Neural networks for machine learning lecture 6a overview of mini-batch gradient descent. 2012;14(8).

  65. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V. et al. Scikit-learn: Machine learning in python. J Mac Learn Res. 2011;12:2825–2830

  66. Sokolova M, Lapalme G. A systematic analysis of performance measures for classification tasks. Information processing & management. 2009;45(4):427–37.

    Article  Google Scholar 

Download references

Acknowledgements

This research is supported by the Agency for Science, Technology and Research (A*STAR), under its AME Programmatic Funding Scheme (Project #A18A2b0046). We also thank the University Grants Commission, Government of India, for supporting this work under the UGC National Fellowship.

Author information

Authors and Affiliations

  1. Department of Information Science and Technology, Anna University, Chennai, 600025, India

    Ashok Kumar J & Tina Esther Trueman

  2. School of Computer Science and Engineering, Nanyang Technological University, Singapore, Singapore

    Erik Cambria

Authors
  1. Ashok Kumar J
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tina Esther Trueman
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erik Cambria
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Erik Cambria.

Ethics declarations

Conflict of Interest

The authors declare that they have no conflict of interest.

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

J, A.K., Trueman, T.E. & Cambria, E. A Convolutional Stacked Bidirectional LSTM with a Multiplicative Attention Mechanism for Aspect Category and Sentiment Detection. Cogn Comput 13, 1423–1432 (2021). https://doi.org/10.1007/s12559-021-09948-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-021-09948-0

Keywords

Search

Navigation