Skip to main content
SpringerLink
Log in

Exploiting bi-directional deep neural networks for multi-domain sentiment analysis using capsule network

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Sentiment analysis (SA) is the computational analysis of the ideas, feelings, and opinions that determines the polarity of the text documents or comments using natural language processing (NLP) and text analyses techniques. The purpose of the multi-domain SA is to train a classifier using an appropriate set of tagged data to reduce the need for large amounts of data on specific domains and to address their data scarcity challenges using existing data in other domains. A combined use of the pre-trained BERT model, convolutional neural network (CNN), bi-directional long short-term memory (LSTM) and gated recurrent unit (GRU) is exploited in the proposed method of this paper for analysing the multi-domain sentiments using capsule network (CapsuleNet). In the proposed model of this paper, the pre-trained BERT (with CNN) and LSTM extracts the proper features for the CapsuleNet. The proposed approach is evaluated using the Dranziera protocol and the experimental results show that the accuracy of the proposed method is improved in comparison with the other basic deep learning-based methods, such as Multi CNN and LSTM. The results of the experiments show the superiority of the proposed method compared to the other similar methods on in-domain and out-of-domain data.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Data availability

Data sharing not applicable to this article as no datasets were generated or analyzed during the current study.

Notes

  1. https://www.tensorflow.org/

  2. https://cntk.ai/pythondocs/

  3. https://github.com/Theano/Theano

References

  1. Atzeni M, Recupero DR (2020) Multi-domain sentiment analysis with mimicked and polarized word embeddings for human–robot interaction. Futur Gener Comput Syst 110:984–999

    Article  Google Scholar 

  2. Bahdanau D, Cho K, Bengio Y (2014) Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:14090473

  3. Basari ASH, Hussin B, Ananta IGP, Zeniarja J (2013) Opinion mining of movie review using hybrid method of support vector machine and particle swarm optimization. Procedia Engineering 53:453–462

    Article  Google Scholar 

  4. Beigi OM, Moattar MH (2021) Automatic construction of domain-specific sentiment lexicon for unsupervised domain adaptation and sentiment classification. Knowl-Based Syst 213:106423

    Article  Google Scholar 

  5. Chauhan A, Babu M, Kandru N, Lokegaonkar S (2018) Empirical study on convergence of capsule networks with various hyperparameters. Virginia Polytechnic Institute and State University Blacksburg, VA, US,

  6. Cho K, Van Merriënboer B, Bahdanau D, Bengio Y (2014) On the properties of neural machine translation: encoder-decoder approaches. arXiv preprint arXiv:14091259

  7. Church K, Hanks P (1989) Word association norms, mutual information and lexicography. I: ACL 27th annual meeting 76–83. Vancouver Halvautomatisk ekserpering av anglisismer i norsk 85

  8. Demotte P, Wijegunarathna K, Meedeniya D, Perera I (2021) Enhanced sentiment extraction architecture for social media content analysis using capsule networks. Multimed Tools Appl: 1–26

  9. Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:181004805

  10. Dragoni M, Petrucci G (2017) A neural word embeddings approach for multi-domain sentiment analysis. IEEE Trans Affect Comput 8(4):457–470

    Article  Google Scholar 

  11. Dragoni M, Petrucci G (2018) A fuzzy-based strategy for multi-domain sentiment analysis. Int J Approx Reason 93:59–73

    Article  MathSciNet  Google Scholar 

  12. Dragoni M, Tettamanzi AG, da Costa PC (2016) DRANZIERA: an evaluation protocol for multi-domain opinion mining. In: Tenth International Conference on Language Resources and Evaluation (LREC 2016), 2016. European Language Resources Association (ELRA), pp 267–272

  13. Fernández-Gavilanes M, Alvarez-López T, Juncal-Martínez J, Costa-Montenegro E, González-Castano FJ (2015) Gti: An unsupervised approach for sentiment analysis in twitter. In: Proceedings of the 9th International Workshop on Semantic Evaluation (SemEval 2015), pp 533–538

  14. Gers FA, Schmidhuber J, Cummins F (2000) Learning to forget: continual prediction with LSTM. Neural Comput 12(10):2451–2471

    Article  Google Scholar 

  15. Ghorbanali A, Sohrabi MK, Yaghmaee F (2022) Ensemble transfer learning-based multimodal sentiment analysis using weighted convolutional neural networks. Inf Process Manag 59(3):102929

    Article  Google Scholar 

  16. Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2016) Deep learning for visual understanding: a review. Neurocomputing 187:27–48

    Article  Google Scholar 

  17. Hemmatian F, Sohrabi MK (2019) A survey on classification techniques for opinion mining and sentiment analysis. Artif Intell Rev 52(3):1495–1545

    Article  Google Scholar 

  18. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  19. Jha V, Savitha R, Shenoy PD, Venugopal K, Sangaiah AK (2018) A novel sentiment aware dictionary for multi-domain sentiment classification. Comput Electric Eng 69:585–597

    Article  Google Scholar 

  20. Kim Y (2014) Convolutional neural networks for sentence classification. arXiv preprint arXiv: 14085882. https://doi.org/10.3115/v1.D14-1181

  21. Krishnakumari K, Sivasankar E, Radhakrishnan S (2020) Hyperparameter tuning in convolutional neural networks for domain adaptation in sentiment classification (HTCNN-DASC). Soft Comput 24(5):3511–3527

    Article  Google Scholar 

  22. LeCun Y, Haffner P, Bottou L, Bengio Y (1999) Object Recognition with Gradient-Based Learning. In: Object recognition with gradient-based learning. Shape, contour and grouping in computer vision. Springer, In, pp 319–345

    Google Scholar 

  23. Li J, Luong M-T, Jurafsky D, Hovy E (2015) When are tree structures necessary for deep learning of representations? arXiv preprint arXiv:150300185

  24. Liu B (2012) Sentiment analysis and opinion mining. Synth Lect Hum Lang Technol 5(1):1–167

    Article  Google Scholar 

  25. Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. Mining text data. Springer, In, pp 415–463

    Book  Google Scholar 

  26. Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: a survey. Ain Shams engineering journal 5(4):1093–1113

    Article  Google Scholar 

  27. Nassif AB, Elnagar A, Shahin I, Henno S (2021) Deep learning for Arabic subjective sentiment analysis: challenges and research opportunities. Appl Soft Comput 98:106836

    Article  Google Scholar 

  28. Otter DW, Medina JR, Kalita JK (2020) A survey of the usages of deep learning for natural language processing. IEEE Trans Neural Netw Learning Syst 32(2):604–624

    Article  MathSciNet  Google Scholar 

  29. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up? Sentiment classification using machine learning techniques. arXiv preprint cs/0205070

  30. Peng H, Ma Y, Poria S, Li Y, Cambria E (2021) Phonetic-enriched text representation for Chinese sentiment analysis with reinforcement learning. Information Fusion 70:88–99

    Article  Google Scholar 

  31. Pröllochs N, Feuerriegel S, Lutz B, Neumann D (2020) Negation scope detection for sentiment analysis: a reinforcement learning framework for replicating human interpretations. Inf Sci 536:205–221

    Article  Google Scholar 

  32. Pucci R, Micheloni C, Foresti GL, Martinel N (2020) Deep interactive encoding with capsule networks for image classification. Multimed Tools Appl 79(43):32243–32258

    Article  Google Scholar 

  33. Rojas-Barahona LM (2016) Deep learning for sentiment analysis. Lang Linguist Compass 10(12):701–719

    Article  Google Scholar 

  34. Routray P, Swain CK, Mishra SP (2013) A survey on sentiment analysis. Int J Comput Appl 76(10):1–8

    Google Scholar 

  35. Sabour S, Frosst N, Hinton GE (2017) Dynamic routing between capsules. Adv Neural Inf Proces Syst 30

  36. Sohrabi MK, Hemmatian F (2019) An efficient preprocessing method for supervised sentiment analysis by converting sentences to numerical vectors: a twitter case study. Multimed Tools Appl 78(17):24863–24882

    Article  Google Scholar 

  37. Tai KS, Socher R, Manning CD (2015) Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:150300075

  38. Tembhurne JV, Diwan T (2021) Sentiment analysis in textual, visual and multimodal inputs using recurrent neural networks. Multimed Tools Appl 80(5):6871–6910

    Article  Google Scholar 

  39. Torabian B (2016) Sentiment classification with case-base approach.

  40. Tripathy A, Anand A, Kadyan V (2022) Sentiment classification of movie reviews using GA and NeuroGA. Multimed Tools Appl:1–21

  41. Turney PD (2002) Thumbs up or thumbs down? Semantic orientation applied to unsupervised classification of reviews. arXiv preprint cs/0212032

  42. Wu F, Yuan Z, Huang Y (2017) Collaboratively training sentiment classifiers for multiple domains. IEEE Trans Knowl Data Eng 29(7):1370–1383

    Article  Google Scholar 

  43. Yadav A, Vishwakarma DK (2020) Sentiment analysis using deep learning architectures: a review. Artif Intell Rev 53(6):4335–4385

    Article  Google Scholar 

  44. Yang SY, Yu Y, Almahdi S (2018) An investor sentiment reward-based trading system using Gaussian inverse reinforcement learning algorithm. Expert Syst Appl 114:388–401

    Article  Google Scholar 

  45. Yang M, Jiang Q, Shen Y, Wu Q, Zhao Z, Zhou W (2019) Hierarchical human-like strategy for aspect-level sentiment classification with sentiment linguistic knowledge and reinforcement learning. Neural Netw 117:240–248

    Article  Google Scholar 

  46. Ye Q, Zhang Z, Law R (2009) Sentiment classification of online reviews to travel destinations by supervised machine learning approaches. Expert Syst Appl 36(3):6527–6535

    Article  Google Scholar 

  47. Yuan Z, Wu S, Wu F, Liu J, Huang Y (2018) Domain attention model for multi-domain sentiment classification. Knowl-Based Syst 155:1–10

    Article  Google Scholar 

  48. Yue C, Cao H, Xu G, Dong Y (2021) Collaborative attention neural network for multi-domain sentiment classification. Appl Intell 51(6):3174–3188

    Article  Google Scholar 

  49. Zagibalov T, Carroll JA (2008) Automatic seed word selection for unsupervised sentiment classification of Chinese text. In: Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pp 1073–1080

  50. Zhao C, Wang S, Li D (2020) Multi-source domain adaptation with joint learning for cross-domain sentiment classification. Knowl-Based Syst 191:105254

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Semnan Branch, Islamic Azad University, Semnan, Iran

    Alireza Ghorbanali & Mohammad Karim Sohrabi

Authors
  1. Alireza Ghorbanali
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Mohammad Karim Sohrabi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mohammad Karim Sohrabi.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interests.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ghorbanali, A., Sohrabi, M.K. Exploiting bi-directional deep neural networks for multi-domain sentiment analysis using capsule network. Multimed Tools Appl 82, 22943–22960 (2023). https://doi.org/10.1007/s11042-023-14449-3

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-023-14449-3

Keywords

Search

Navigation