Skip to main content
SpringerLink
Log in

A Novel Deep Learning Language Model with Hybrid-GFX Embedding and Hyperband Search for Opinion Analysis

  • Original Research
  • Published:
SN Computer Science Aims and scope Submit manuscript

Abstract

Policies, legislation, surveillance, monitoring, direction, and enforcement, are heavily influenced by public opinion or emotion. Due to the increase in electronic data generation, it has been forced to do an automatic analysis of this opinion or feelings termed as opinion analysis. To process massive volumes of data, deep learning is now trending. Word embeddings serve an essential role of feature representatives in deep understanding. The present paper offers a novel deep learning architecture that represents hybrid embedding that deals with polysemy, semantic, and syntactic problems in a language representation. The effectiveness of a deep learning model is extremely sensitive to using hyperparameters. Here, the proposed a novel Hybrid-GFX–Attention–BiGRU–CNN model with a hyperband language model. Hyperband search is used to find optimal values for the model's hyperparameters. To justify classification results, statistical and graphical approaches have been used. We analyzed the model's efficacy using the MR and Hate speech data sets. The model’s performance is quite promising compared with existing state-of-the-art architectures.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Data availability

The dataset generated during and/or analysed during the current study are available from the corresponding author on reasonable request.

Notes

  1. https://ai.google/research/teams/brain/pair.

References

  1. Kinra A, Beheshti-Kashi S, Buch R, Nielsen TAS, Pereira F. Examining the potential of textual big data analytics for public policy decision-making: A case study with driverless cars in Denmark. Transp Policy. 2020;98:68–78.

    Article  Google Scholar 

  2. Yenkar PP, Sawarkar SD. A novel ensemble approach based on MCC and MCDM methods for prioritizing tweets mentioning urban issues in smart cities. Kybernetes. 2022. https://doi.org/10.1108/K-08-2021-0785.

    Article  Google Scholar 

  3. Kauffmann E, Peral J, Gil D, Ferrández A, Sellers R, Mora H. Managing marketing decision-making with sentiment analysis: An evaluation of the main product features using text data mining. Sustainability. 2019;11(15):4235.

    Article  Google Scholar 

  4. Fan ZP, Xi Y, Li Y. Supporting consumers’ purchase decisions: a comprehensive method for selecting desirable online products. Kybernetes. 2018;47(4):689–715.

    Article  Google Scholar 

  5. Gupta MV, Vaikole S, Oza AD, Patel A, Burduhos-Nergis DP, Burduhos-Nergis DD. Audio-visual stress classification using cascaded RNN-LSTM networks. Bioengineering. 2022;9(10):510.

    Article  Google Scholar 

  6. Boukabous M, Azizi M. Review of learning-based techniques of sentiment analysis for security purposes. In: The proceedings of the third international conference on smart city applications. Cham: Springer International Publishing, 2020. p. 96–109.

  7. Ragini JR, Anand PR, Bhaskar V. Big data analytics for disaster response and recovery through sentiment analysis. Int J Inf Manag. 2018;42:13–24.

    Article  Google Scholar 

  8. Malawani AD, Nurmandi A, Purnomo EP, Rahman T. Social media in aid of post-disaster management. Transform Gov People Process Policy. 2020;14(2):237–60.

    Google Scholar 

  9. Salur MU, Aydin I. A novel hybrid deep learning model for sentiment classification. IEEE Access. 2020;8:58080–93.

    Article  Google Scholar 

  10. Liu B. Sentiment analysis: mining sentiments, opinions, and emotions. Cambridge: Cambridge University; 2015.

    Book  Google Scholar 

  11. Pang B, Lee L. A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts. arXiv:cs/0409058 [preprint]. 2004.

  12. Goldberg Y. A primer on neural network models for natural language processing. J Artif Intell Res. 2016;57:345–420.

    Article  MathSciNet  MATH  Google Scholar 

  13. Zhang L, Wang S, Liu B. Deep learning for sentiment analysis: A survey. Wiley Interdiscip Rev Data Mining Knowl Discov. 2018;8(4): e1253.

    Article  Google Scholar 

  14. Teng Z, Vo DT, Zhang Y. Context-sensitive lexicon features for neural sentiment analysis. In: Proceedings of the 2016 conference on empirical methods in natural language processing. 2016.

  15. Al-Moslmi T, Omar N, Abdullah S, Albared M. Approaches to cross-domain sentiment analysis: A systematic literature review. IEEE access. 2017;5:16173–92.

    Article  Google Scholar 

  16. Yan K, Zhong C, Ji Z, Huang J. Semi-supervised learning for early detection and diagnosis of various air handling unit faults. Energy Build. 2018;181:75–83.

    Article  Google Scholar 

  17. Lu H, Yang L, Yan K, Xue Y, Gao Z. A cost-sensitive rotation forest algorithm for gene expression data classification. Neurocomputing. 2017;228:270–6.

    Article  Google Scholar 

  18. Neethu MS, Rajasree R. Sentiment analysis in Twitter using machine learning techniques. In: 2013 fourth international conference on computing, communications, and networking technologies (ICCCNT). IEEE, 2013. p. 1–5.

  19. Xia H, Yang Y, Pan X, Zhang Z, An W. Sentiment analysis for online reviews using conditional random fields and support vector machines. Electron Comm Res. 2020;20:343–60.

    Article  Google Scholar 

  20. Qu L, Ifrim G, Weikum G. The bag-of-opinions method for review rating prediction from sparse text patterns. In Proceedings of the 23rd international conference on computational linguistics (Coling 2010). 2010. p. 913–921.

  21. Campos V, Jou B, Giro-i-Nieto X. From pixels to sentiment: Fine-tuning CNNs for visual sentiment prediction. Image Vis Comput. 2017;65:15–22.

    Article  Google Scholar 

  22. Marasek K. Deep belief neural networks and bidirectional long-short term memory hybrid for speech recognition. Arch Acoust. 2015;40(2):191–5.

    Article  Google Scholar 

  23. Schmidhuber J. Deep learning neural networks: An overview. Neural Netw. 2015;61:85–117.

    Article  Google Scholar 

  24. Ghannay S, Favre B, Esteve Y, Camelin N. Word embedding evaluation and combination. In: Proceedings of the tenth international conference on language resources and evaluation (LREC'16). 2016. p. 300–305.

  25. Wang J, Zhang Y, Yu LC, Zhang X. Contextual sentiment embeddings via bi-directional GRU language model. Knowl-Based Syst. 2022;235: 107663.

    Article  Google Scholar 

  26. Reynolds K, Kontostathis A, Edwards L. Using machine learning to detect cyberbullying. In: 2011 10th international conference on machine learning and applications and workshops (vol. 2). IEEE, 2011. p. 241–244.

  27. Prabha MI, Srikanth GU. Survey of sentiment analysis using deep learning techniques. In: 2019 1st international conference on innovations in information and communication technology (ICIICT). IEEE, 2019. p. 1–9.

  28. Hassan A, Mahmood A. Convolutional recurrent deep learning model for sentence classification. IEEE Access. 2018;6:13949–57.

    Article  Google Scholar 

  29. Jihan N, Senarath Y, Ranathunga S. Aspect extraction from customer reviews using convolutional neural networks. In: 2018 18th international conference on advances in ICT for emerging regions (ICTer). IEEE, 2018. p. 215–220.

  30. Çano E, Morisio M. A deep learning architecture for sentiment analysis. In: Proceedings of the international conference on geoinformatics and data analysis. 2018. p. 122–126.

  31. Huang C, Liu G. Sentiment analysis of network comments based on GCNN. In: Proceedings of the 2018 2nd international conference on computer science and artificial intelligence. 2018. p. 409–413.

  32. Cheng LC, Tsai SL. Deep learning for automated sentiment analysis of social media. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining. 2019. p. 1001–1004.

  33. Chiong R, Fan Z, Hu Z, Adam MT, Lutz B, Neumann D. A sentiment analysis-based machine learning approach for financial market prediction via news disclosures. In: Proceedings of the genetic and evolutionary computation conference companion. 2018. p. 278–279.

  34. Sun H, Jiang T, Dai Y. Sentiment analysis of commodity reviews based on multilayer LSTM network. In: Proceedings of the international conference on artificial intelligence, information processing and cloud computing. 2019. p. 1–5.

  35. Beseiso M, Elmousalami H. Subword attentive model for Arabic sentiment analysis: A deep learning approach. ACM Trans Asian Low-Resour Lang Inf Process (TALLIP). 2020;19(2):1–17.

    Article  Google Scholar 

  36. Rakhmanov O. On validity of sentiment analysis scores and development of classification model for student-lecturer comments using weight-based approach and DEEP LEARNING. In: Proceedings of the 21st annual conference on information technology education. 2020. p. 174–179.

  37. Alcamo T, Cuzzocrea A, Bosco GL, Pilato G, Schicchi D. Analysis and comparison of deep learning networks for supporting sentiment mining in text corpora. In: Proceedings of the 22nd international conference on information integration and web-based applications & services. 2020. p. 91–96.

  38. Pota M, Ventura M, Catelli R, Esposito M. An effective BERT-based pipeline for Twitter sentiment analysis: A case study in Italian. Sensors. 2020;21(1):133.

    Article  Google Scholar 

  39. Bollegala D, O'Neill J. A survey on word meta-embedding learning. arXiv:2204.11660 [Preprint]. 2022.

  40. Alharbi AI, Smith P, Lee M. Enhancing contextualized language models with static character and word embeddings for emotional intensity and sentiment strength detection in Arabic tweets. Procedia Comput Sci. 2021;189:258–65.

    Article  Google Scholar 

  41. Jawale S, Sawarkar SD. Sentiment analysis and vector embedding: A comparative study. In: Smart trends in computing and communications: Proceedings of SmartCom 2022. Singapore: Springer Nature Singapore, 2022. p. 311–321.

  42. Hameed Z, Garcia-Zapirain B. Sentiment classification using a single-layered BiLSTM model. IEEE Access. 2020;8:73992–4001.

    Article  Google Scholar 

  43. Gupta P, Jaggi M. Obtaining better static word embeddings using contextual embedding models. arXiv:2106.04302 [Preprint]. 2021.

  44. Dorle S, Pise NN. Sentiment analysis methods and approach: Survey. Int J Innov Comput Sci Eng. 2017;4(6):7–11.

    Google Scholar 

  45. Bojanowski P, Grave E, Joulin A, Mikolov T. Enriching word vectors with subword information. Trans Assoc Comput Linguist. 2017;5:135–46.

    Article  Google Scholar 

  46. Pennington J, Socher R, Manning CD. GloVe: Global vectors for word representation. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014. p. 1532–1543.

  47. Dai Z, Yang Z, Yang Y, Carbonell J, Le QV, Salakhutdinov R. Transformer-xl: Attentive language models beyond a fixed-length context. arXiv:1901.02860 [Preprint]. 2019.

  48. Tripepi G, Jager KJ, Dekker FW, Zoccali C. Linear and logistic regression analysis. Kidney Int. 2008;73(7):806–10.

    Article  Google Scholar 

  49. Scholkopf B. Support vector machines: a practical consequence of learning theory. IEEE Intell Syst. 1998. https://doi.org/10.1041/X4018s-1998.

    Article  Google Scholar 

  50. Kuvalekar A, Manchewar S, Mahadik S, Jawale S. House price forecasting using machine learning. In: Proceedings of the 3rd international conference on advances in science & technology (ICAST). 2020.

  51. Yoon K. Convolutional neural networks for sentence classification [OL]. arXiv:1408.5882 [Preprint]. 2014.

  52. Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.

    Article  Google Scholar 

  53. Basiri ME, Nemati S, Abdar M, Cambria E, Acharya UR. ABCDM: An attention-based bidirectional CNN-RNN deep model for sentiment analysis. Fut Gener Comput Syst. 2021;115:279–94.

    Article  Google Scholar 

  54. Li L, Yang L, Zeng Y. Improving sentiment classification of restaurant reviews with an attention-based bi-GRU neural network. Symmetry. 2021;13(8):1517.

    Article  Google Scholar 

  55. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, et al. Attention is all you need. Adv Neural Inf Process Syst. 2017;30:5998–6008.

    Google Scholar 

  56. Sukhbaatar S, Weston J, Fergus R. End-to-end memory networks. Adv Neural Inf Process Syst 2015;28.

  57. He R, McAuley J. Fusing similarity models with Markov chains for a sparse sequential recommendation. In: 2016 IEEE 16th international conference on data mining (ICDM). IEEE, 2016. p. 191–200.

  58. Wall ME, Rechtsteiner A, Rocha LM. Singular value decomposition and principal component analysis. In: Berrar DP, Dubitzky W, Granzow M, editors. A practical approach to microarray data analysis. Boston: Springer; 2003. p. 91–109.

    Chapter  Google Scholar 

  59. Goodfellow I, Bengio Y, Courville A. Deep learning. Cambridge: MIT Press; 2016.

    MATH  Google Scholar 

  60. Li L, Jamieson K, DeSalvo G, Rostamizadeh A, Talwalkar A. Hyperband: A novel bandit-based approach to hyperparameter optimization. J Mach Learn Res. 2017;18(1):6765–816.

    MathSciNet  MATH  Google Scholar 

  61. Feurer M, Hutter F. Hyperparameter optimization. In: Hutter F, Kotthoff L, Vanschoren J, editors. Automated machine learning: Methods, systems, challenges. The Springer series on challenges in machine learning. Cham: Springer; 2019. p. 3–33.

    Chapter  Google Scholar 

  62. Prechelt L. Neural networks: Tricks of the trade. Lecture notes in computer science, 1524. Berlin: Springer; 1998. p. 53–67.

    Google Scholar 

  63. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: A simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.

    MathSciNet  MATH  Google Scholar 

  64. Palomino MA, Aider F. Evaluating the effectiveness of text pre-processing in sentiment analysis. Appl Sc. 2022;12(17):8765.

    Article  Google Scholar 

  65. McInnes L, Healy J, Melville J. Umap: Uniform manifold approximation and projection for dimension reduction. arXiv:1802.03426 [Preprint]. 2018.

  66. Ribeiro MT, Singh S, Guestrin C. "Why should I trust you?" Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. 2016. p. 1135–1144.

  67. Chatterjee A, Gupta U, Chinnakotla MK, Srikanth R, Galley M, Agrawal P. Understanding emotions in text using DEEP LEARNING and big data. Comput Hum Behav. 2019;93:309–17.

    Article  Google Scholar 

  68. Basile V, Bosco C, Fersini E, Nozza D, Patti V, Pardo FMR, et al. Semeval-2019 task 5: Multilingual detection of hate speech against immigrants and women in twitter. In: Proceedings of the 13th international workshop on semantic evaluation. 2019. p. 54–63.

  69. Priyadarshini I, Cotton C. A novel LSTM–CNN–grid search-based deep neural network for sentiment analysis. J Supercomput. 2021;77(12):13911–32.

    Article  Google Scholar 

  70. Rehman AU, Malik AK, Raza B, Ali W. A hybrid CNN-LSTM model for improving the accuracy of movie reviews sentiment analysis. Multimed Tools Appl. 2019;78(18):26597–613.

    Article  Google Scholar 

  71. Pimpalkar A. MBiLSTMGloVe: Embedding GloVe knowledge into the corpus using multi-layer BiLSTM DEEP LEARNING model for social media sentiment analysis. Expert Syst Appl. 2022;203: 117581.

    Article  Google Scholar 

  72. Long Y, Li Y, Luo J, Miao C, Fu J. MCP-LSTM network for sentence-level sentiment classification. In: 2019 International conference on virtual reality and visualization (ICVRV). IEEE, 2019. p. 124–128.

  73. Khasanah IN. Sentiment classification using fasttext embedding and deep learning model. Procedia Comput Sci. 2021;189:343–50.

    Article  Google Scholar 

  74. Hengle A, Kshirsagar A, Desai S, Marathe M. Combining context-free and contextualized representations for Arabic sarcasm detection and sentiment identification. arXiv:2103.05683 [preprint]. 2021.

  75. Thakur NR, Talwai P, Jawale S. Ant colony optimization for load balancing and congestion control. Int J Syst Algorithms Appl. 2012;2(9):5.

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Datta Meghe College of Engineering, Airoli, Navi Mumbai, Maharashtra, India

    Shila Jawale & S. D. Sawarkar

Authors
  1. Shila Jawale
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. S. D. Sawarkar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shila Jawale.

Ethics declarations

Conflict of interest

The authors declare no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article is part of the topical collection “Machine Intelligence and Smart Systems” guest edited by Manish Gupta and Shikha Agrawal.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Jawale, S., Sawarkar, S.D. A Novel Deep Learning Language Model with Hybrid-GFX Embedding and Hyperband Search for Opinion Analysis. SN COMPUT. SCI. 4, 759 (2023). https://doi.org/10.1007/s42979-023-02236-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s42979-023-02236-8

Keywords

Search

Navigation