Abstract
Identifying sentiment polarity of a document is a building block of sentiment analysis and natural language processing tasks, and it aims to automate the prediction of a user’s sentiment orientation in the document about a product, on assumption that the document expresses a sentiment on a single product. In general, supervised machine learning models like support vector machine and recently fast-growing deep neural networks method have been extensively used as a sentiment learning approach. Although some neural network-based models learn text features without feature engineering, most of them only focus on extracting semantic representations from single words and rarely consider the contexts attributed to the correlation between words and sentences. In this paper, we propose a novel neural network model to capture the context information from texts. Our model builds a hybrid neural network model using convolutional neural networks and long short-term memory for word context extraction and document representation, respectively. On this basis, user’s and product’s information can be incorporated into the model. The experimental results show the competitive performance of our model, compared to all state-of-the-art methods.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retrieval 2(12):1–135
Pang B, Lee L (2002) Thumbs up? Sentiment classification using machine learning techniques. In: EMNLP, pp 79–86
Cambria E (2011) Affective computing and sentiment analysis, no. 2, vol 45. Springer, Netherlands, pp 102–107
Hatzivassiloglou V, McKeown K (1997) Predicting the semantic orientation of adjectives. In: Proceedings of ACL
Taboada M, Brooke J, Tofiloski M, Voll K, Stede M (2011) Lexicon-based methods for lexicon-based methods for sentiment analysis. In: ACL
Zhang H, Cao X, H J, C T (2017) Object-level video advertising: an optimization framework. IEEE Trans Ind Inform 13(2):520–531
Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: International conference on neural information processing systems, pp 1097–1105
Amodei D, Anubhai R, Battenberg E, Case C, Casper J, Catanzaro B, Chen J, Chrzanowski M, Coates A, Diamos G (2015) Deep speech 2: end-to-end speech recognition in English and Mandarin. Comput Sci 127:123–132
Zadeh A, Chen M, Poria S, Cambria E, Morency L-P (2017) Tensor fusion network for multimodal sentiment analysis. In: EMNLP, pp 1103–1114
Zhang H, Li J, J Y, Y H (2017) Understanding subtitles by character-level sequence-to-sequence learning. IEEE Trans Ind Inform 13(2):616–624
Kim Y (2014) Convolutional neural networks for sentence classification. In: Proceedings of EMNLP
Socher R, Pennington J, Huang EH, Ng AY, Manning CD (2011) Semi-supervised recursive autoencoders for predicting sentiment distributions. In: Proceedings of EMNLP, pp 151–161
Tang D, Qin B, Liu T (2015) Learning semantic representations of users and products for document level sentiment classification. In: Proceedings of ACL
Chen H, Sun M, Tu C, Lin Y, Liu Z (2016) Neural sentiment classification with user and product attention. In: proceedings of EMNLP
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781
Maas AL, Daly RE, Pham PT, Huang D, Ng AY, Potts C (2011) Learning word vectors for sentiment analysis. In: Proceedings of ACL, pp 142–150
Tang D, Qin B, Liu T (2015) Document modeling with gated recurrent neural network for sentiment classification. In: Proceedings of EMNLP, pp 1422–1432
Tang D, Wei F, Yang N, Zhou M, Liu T, Qin B (2014) Learning sentiment-specific word embedding for twitter sentiment classification. In: Proceedings of ACL, pp 1555–1565
Labutov I, Lipson H (2013) Re-embedding words. In: Annual meeting of the association for computational linguistics
Johnson R, Zhang T (2014) Effective use of word order for text categorization with convolutional neural networks. arXiv preprint arXiv:1412.1058
Zhang X, LeCun Y (2015) Text understanding from scratch. arXiv:1502.01710v5
Bhatia P, Ji Y, Eisenstein J (2015) Better document-level sentiment analysis from RST discourse parsing. In: Proceedings of EMNLP
Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E (2016) Hierarchical attention networks for document classification. In: Proceedings NAACL
Shi B, Bai X, Yao C (2015) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition. IEEE Trans Pattern Anal Mach Intell 39(11):2298–2304
Frege G (1892) On sense and reference. In: Ludlow
Lipton ZC, Kale DC, Elkan C, Wetzell R (2015) Learning to diagnose with lstm recurrent neural networks. arXiv preprint arXiv:1511.03677
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980 [cs.LG]
Le QV, Mikolov T (2014) Distributed representations of sentences and documents. In: Proceedings of ICML
Acknowledgements
This work was supported by the National Social Science Foundation of China (No. 17BXW071) and the Technology Program of Lanzhou Science and Technology Bureau (No. 214162). P. Li acknowledge SWPU Innovation Team “Data Intelligence” Funding (No. 2015CXTD06) and NFSC (No. 81373531).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
We declare that we do not have any commercial or associative interest that represents a conflict of interest in connection with the work submitted.
Rights and permissions
About this article
Cite this article
Han, H., Bai, X. & Li, P. Augmented sentiment representation by learning context information. Neural Comput & Applic 31, 8475–8482 (2019). https://doi.org/10.1007/s00521-018-3698-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00521-018-3698-4