Skip to main content
SpringerLink
Log in

A Deep Recursive Neural Network Model for Fine-Grained Opinion Classification

  • Conference paper
  • First Online:
Recent Trends in Image Processing and Pattern Recognition (RTIP2R 2018)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1037))

Abstract

In recent times, deep neural networks (DNN) have acquired greater significance in providing solutions to many deep learning tasks. Particularly, recursive neural networks (RNN) have been efficiently utilized in exploring semantic compositions for natural language content represented with structured formats (e.g. parse-trees). Despite the fact that RNN are deep in structure, yet they fail to exhibit hierarchical representations observed in traditional deep feed-forward networks (DFNN) and also in revolutionary deep recurrent neural networks (DRcNN). However, the notion of depth can be incorporated through stacking multiple recursive layers, which results in deep recursive neural networks (DRNN). On the other hand, enhanced word spaces offer added benefits in capturing fine-grained semantic regularities. In this paper, we address the problem of fine-grained opinion classification using DRNN and word embeddings. Furthermore, the efficiency of DRNN model is estimated through the conduction of a series of experiments over several opinion datasets. The results report that the proposed DRNN architecture achieves better prediction rate for fine-grained classification when compared with conventional shallow counterparts that employ similar parameters.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://nlp.stanford.edu/sentiment/code.html.

  2. 2.

    https://github.com/orenmel/context2vec.

References

  1. Wang, H., Can, D., Kazemzadeh, A., Bar, F., Narayanan, S.: A system for real-time Twitter sentiment analysis of 2012 US presidential election cycle. In: Proceedings of the ACL 2012 System Demonstrations, pp. 115–120 (2012)

    Google Scholar 

  2. Liu, W., Wang, Z., Liu, X., Zeng, N., Liu, Y., Alsaadi, F.E.: A survey of deep neural network architectures and their applications. Neurocomputing 234, 11–26 (2017)

    Article  Google Scholar 

  3. Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP 2013 (2013)

    Google Scholar 

  4. Irsoy, O., Cardie, C.: Deep recursive neural networks for compositionality in language. In: Advances in Neural Information Processing Systems, pp. 1–4 (2014)

    Google Scholar 

  5. Bengio, Y.: Learning deep architectures for AI. Found. Trends Mach. Learn. 2(1), 1–127 (2009)

    Article  Google Scholar 

  6. Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256 (2010)

    Google Scholar 

  7. Gori, M., Maggini, M., Sarti, L.: A recursive neural network model for processing directed acyclic graphs with labeled edges. In: Proceedings of the International Joint Conference on Neural Networks (2003)

    Google Scholar 

  8. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–221 (1990)

    Article  Google Scholar 

  9. Pascanu, R., Gulcehre, C., Cho, K., Bengio, Y.: How to construct deep recurrent neural networks. arXiv preprint arXiv:1312.6026 (2013)

  10. Pascanu, R., Mikolov, T., Bengio, Y.: On the difficulty of training recurrent neural networks. In: ICML-2013 (2013)

    Google Scholar 

  11. LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  12. Severyn, A., Moschitti, A.: Twitter sentiment analysis with deep convolutional neural networks. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR-15), pp. 959–962 (2015)

    Google Scholar 

  13. Poria, S., Cambria, E., Gelbukh, A.: Aspect extraction for opinion mining with a deep convolutional neural network. Knowl.-Based Syst. 108, 42–49 (2016)

    Article  Google Scholar 

  14. Ruder, S., Ghaffari, P., Breslin, J.G.: INSIGHT-1 at SemEval-2016 Task 5: deep learning for multilingual aspect-based sentiment analysis. In: Proceedings of SemEval-2016, pp. 330–336 (2016)

    Google Scholar 

  15. Poria, S., Cambria, E., Hazarika, D., Vij, P.: A deeper look into sarcastic tweets using deep convolutional neural networks. In: Proceedings of COLING 2016 (2016)

    Google Scholar 

  16. Ukil, S., Ghosh, S., Obaidullah, Sk.Md., Santosh, K.C., Roy, K., Das, N.: Deep learning for word-level handwritten Indic script identification. Cornell University Library (arXiv:1801.01627) (2018)

  17. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)

    Article  Google Scholar 

  18. Pascanu, R., Gulcehre, C., Cho, K., Bengio, Y.: How to construct deep recurrent neural networks. In: Proceedings of ICLR 2014, pp. 1–10 (2014)

    Google Scholar 

  19. İrsoy, O., Cardie, C.: Modeling compositionality with multiplicative recurrent neural networks. In: Proceedings of ICLR 2015 (2015)

    Google Scholar 

  20. Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. Comput. Intell. Mag. 13, 55–75 (2018)

    Article  Google Scholar 

  21. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  22. Wang, X., Liu, Y., Sun, C., Wang, B., Wang, X.: Predicting polarities of tweets by composing word-embeddings with long short-term memory. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, pp. 1343–1353 (2015)

    Google Scholar 

  23. Graves, A., Mohamed, A., Hinton, G.: Speech recognition with deep recurrent neural networks. In: 2013 IEEE International Conference on Acoustics, Speech and Signal Processing (2013)

    Google Scholar 

  24. Bowman, S.R., Potts, C., Manning, C.D.: Recursive neural networks can learn logical semantics. In: Proceedings of the third Workshop on Continuous Vector Space Models and their Compositionality (2015)

    Google Scholar 

  25. Tang, D., Qin, B., Liu, T.: Aspect level sentiment classification with deep memory network. In: Proceeding of EMNLP 2016 (2016)

    Google Scholar 

  26. Liu, S., Yang, N., Li, M., Zhou, M.: A recursive recurrent neural network for statistical machine translation. In: Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pp. 1491–1500 (2014)

    Google Scholar 

  27. Hassan, A., Mahmood, A.: Convolutional recurrent deep learning model for sentence classification. IEEE Access 6, 13949–13957 (2018)

    Article  Google Scholar 

  28. Li, Z., Huang, J., Zhou, Z., Zhang, H., Chang, S., Huang, Z.: LSTM-based deep learning models for answer ranking. In: 2016 IEEE First International Conference on Data Science in Cyberspace (DSC) (2016)

    Google Scholar 

  29. Van, V.D., Thai, T., Nghiem, M.-Q.: Combining convolution and recursive neural networks for sentiment analysis. In: Proceedings of the Eighth International Symposium on Information and Communication Technology 2017. ACM (2018)

    Google Scholar 

  30. Socher, R., Lin, C.C.-Y., Ng, A.Y., Manning, C.D.: Parsing natural scenes and natural language with recursive neural networks. In: Proceedings of the 28th International Conference on International Conference on Machine Learning (ICML 2011), pp. 129–136 (2011)

    Google Scholar 

  31. Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier networks. In: Proceedings of the 14th International Conference on Artificial Intelligence and Statistics, vol. 15, pp. 315–323 (2011)

    Google Scholar 

  32. Mikolov, T., Sutskever, I., Chen, K., Corrado, G., Dean, J.: Distributed representations of Words and Phrases and their compositionality. arXiv:1310.4546, pp. 1–9 (2013)

  33. Pennington, J., Socher, R., Manning, C.D.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  34. Liu, P., Qiu, X., Huang, X.: Learning context-sensitive word-embeddings with neural tensor skip-gram model. In: Proceedings of the 24th International Conference on Artificial Intelligence (IJCAI 2015), pp. 1284–1290 (2015)

    Google Scholar 

  35. Melamud, O., Goldberger, J., Dagan, I.: Context2vec: learning generic context embedding with bidirectional LSTM. In: Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning (CoNLL), pp 51–61 (2016)

    Google Scholar 

  36. Srivastava, N.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  37. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)

    MathSciNet  MATH  Google Scholar 

  38. Socher, R., Huval, B., Manning, C.D., Ng, A.Y.: Semantic compositionality through recursive matrix-vector spaces. In: Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL 2012), pp. 1201–1211 (2012)

    Google Scholar 

  39. Kalchbrenner, N., Grefenstette, E., Blunsom, P.: A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188 (2014)

Download references

Author information

Authors and Affiliations

  1. Department of Computer Applications, Basaveshwar Engineering College, Bagalkot, 587 102, India

    Ramesh S. Wadawadagi

  2. Department of Computer Science and Engineering, Basaveshwar Engineering College, Bagalkot, 587 102, India

    Veerappa B. Pagi

Authors
  1. Ramesh S. Wadawadagi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Veerappa B. Pagi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ramesh S. Wadawadagi .

Editor information

Editors and Affiliations

  1. Department of Computer Science, University of South Dakota, Vermillion, SD, USA

    K. C. Santosh

  2. Solapur University, Solapur, India

    Ravindra S. Hegadi

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Wadawadagi, R.S., Pagi, V.B. (2019). A Deep Recursive Neural Network Model for Fine-Grained Opinion Classification. In: Santosh, K., Hegadi, R. (eds) Recent Trends in Image Processing and Pattern Recognition. RTIP2R 2018. Communications in Computer and Information Science, vol 1037. Springer, Singapore. https://doi.org/10.1007/978-981-13-9187-3_54

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-9187-3_54

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-9186-6

  • Online ISBN: 978-981-13-9187-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation