Skip to main content
Springer Nature Link
Log in

Improving Document-Level Sentiment Classification with User-Product Gated Network

  • Conference paper
  • First Online:
Web and Big Data (APWeb-WAIM 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12858))

  • 1636 Accesses

Abstract

Document-level sentiment classification is a fundamental task in Natural Language Processing (NLP). Previous studies have demonstrated the importance of personalized sentiment classification by taking user preference and product characteristics on the sentiment ratings into consideration. The state-of-the-art approaches incorporate such information via attention mechanism, where the attention weights are calculated after the texts are encoded into the low-dimensional vectors with LSTM-based models. However, user and product information may be discarded in the process of generating the semantic representations. In this paper, we propose a novel User-Product gated LSTM network (UP-LSTM), which incorporates user and product information into LSTM cells at the same time of generating text representations. Therefore, UP-LSTM can dynamically produce user- and product-aware contextual representations of texts. Moreover, we devise another version of it to improve the training efficiency. We conduct a comprehensive evaluation with three real world datasets. Experimental results show that our model outperforms previous approaches by an obvious margin.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Adhikari, A., Ram, A., Tang, R., Lin, J.: Docbert: BERT for document classification. CoRR arXiv:1904.08398 (2019)

  2. Amplayo, R.K., Kim, J., Sung, S., Hwang, S.: Cold-start aware user and product attention for sentiment classification. In: ACL, pp. 2535–2544 (2018)

    Google Scholar 

  3. Chen, H., Sun, M., Tu, C., Lin, Y., Liu, Z.: Neural sentiment classification with user and product attention. In: EMNLP, pp. 1650–1659 (2016)

    Google Scholar 

  4. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: NAACL-HLT, pp. 4171–4186 (2019)

    Google Scholar 

  5. Ding, X., Liu, B., Yu, P.S.: A holistic lexicon-based approach to opinion mining. In: WSDM, pp. 231–240 (2008)

    Google Scholar 

  6. Feng, S., Wang, Y., Liu, L., Wang, D., Yu, G.: Attention based hierarchical LSTM network for context-aware microblog sentiment classification. World Wide Web 22(1), 59–81 (2018). https://doi.org/10.1007/s11280-018-0529-6

  7. Greff, K., Srivastava, R.K., Koutník, J., Steunebrink, B.R., Schmidhuber, J.: LSTM: A search space odyssey. IEEE Trans. Neural Netw. Learning Syst. 28(10), 2222–2232 (2017)

    Google Scholar 

  8. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  9. Hsu, W.-Y., Hsu, H.-H., Tseng, V.S.: Discovering negative comments by sentiment analysis on web forum. World Wide Web 22(3), 1297–1311 (2018). https://doi.org/10.1007/s11280-018-0561-6

  10. Kim, Y.: Convolutional neural networks for sentence classification. In: EMNLP, pp. 1746–1751 (2014)

    Google Scholar 

  11. Lei, Z., Yang, Y., Yang, M.: Sentiment lexicon enhanced attention-based LSTM for sentiment classification. In: AAAI, pp. 8105–8106 (2018)

    Google Scholar 

  12. Liu, P., Qiu, X., Chen, X., Wu, S., Huang, X.: Multi-timescale long short-term memory neural network for modelling sentences and documents. In: EMNLP, pp. 2326–2335 (2015)

    Google Scholar 

  13. Luo, L., Ao, X., Pan, F., Wang, J., Zhao, T., Yu, N., He, Q.: Beyond polarity: Interpretable financial sentiment analysis with hierarchical query-driven attention. In: IJCAI, pp. 4244–4250 (2018)

    Google Scholar 

  14. Ma, Y., Li, Q.: A weakly-supervised extractive framework for sentiment-preserving document summarization. World Wide Web 22(4), 1401–1425 (2018). https://doi.org/10.1007/s11280-018-0591-0

  15. Mikolov, T., Sutskever, I., Chen, K., Corrado, G.S., Dean, J.: Distributed representations of words and phrases and their compositionality. In: NIPS, pp. 3111–3119 (2013)

    Google Scholar 

  16. Qian, Q., Huang, M., Lei, J., Zhu, X.: Linguistically regularized LSTM for sentiment classification. In: ACL, pp. 1679–1689 (2017)

    Google Scholar 

  17. Qu, L., Ifrim, G., Weikum, G.: The bag-of-opinions method for review rating prediction from sparse text patterns. In: COLING, pp. 913–921 (2010)

    Google Scholar 

  18. Schouten, K., Frasincar, F.: Survey on aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng. 28(3), 813–830 (2016)

    Article  Google Scholar 

  19. Shen, J., et al.: Dual memory network model for sentiment analysis of review text. Knowl. Based Syst. 188, 105004 (2020)

    Article  Google Scholar 

  20. Sutskever, I., Vinyals, O., Le, Q.V.: Sequence to sequence learning with neural networks. In: NIPS, pp. 3104–3112 (2014)

    Google Scholar 

  21. Tang, D., Qin, B., Liu, T.: Document modeling with gated recurrent neural network for sentiment classification. In: EMNLP, pp. 1422–1432 (2015)

    Google Scholar 

  22. Tang, D., Qin, B., Liu, T.: Learning semantic representations of users and products for document level sentiment classification. In: ACL, pp. 1014–1023 (2015)

    Google Scholar 

  23. Tian, B., Zhang, Y., Wang, J., Xing, C.: Hierarchical inter-attention network for document classification with multi-task learning. In: IJCAI, pp. 3569–3575 (2019)

    Google Scholar 

  24. Wang, J., Lin, C., Li, M., Zaniolo, C.: Boosting approximate dictionary-based entity extraction with synonyms. Inf. Sci. 530, 1–21 (2020)

    Article  Google Scholar 

  25. Wang, J., Lin, C., Zaniolo, C.: Mf-join: Efficient fuzzy string similarity join with multi-level filtering. In: ICDE, pp. 386–397 (2019)

    Google Scholar 

  26. Wang, J., Wang, Z., Zhang, D., Yan, J.: Combining knowledge with deep convolutional neural networks for short text classification. In: IJCAI, pp. 2915–2921 (2017)

    Google Scholar 

  27. Wang, Y., Huang, M., Zhu, X., Zhao, L.: Attention-based LSTM for aspect-level sentiment classification. In: EMNLP, pp. 606–615 (2016)

    Google Scholar 

  28. Wu, F., Huang, Y.: Personalized microblog sentiment classification via multi-task learning. In: AAAI, pp. 3059–3065 (2016)

    Google Scholar 

  29. Wu, J., Zhang, Y., Wang, J., Lin, C., Fu, Y., Xing, C.: Scalable metric similarity join using mapreduce. In: ICDE, pp. 1662–1665 (2019)

    Google Scholar 

  30. Wu, Z., Dai, X., Yin, C., Huang, S., Chen, J.: Improving review representations with user attention and product attention for sentiment classification. In: AAAI, pp. 5989–5996 (2018)

    Google Scholar 

  31. Xing, B., Liao, L., Song, D., Wang, J., Zhang, F., Wang, Z., Huang, H.: Earlier attention? aspect-aware LSTM for aspect-based sentiment analysis. In: IJCAI, pp. 5313–5319 (2019)

    Google Scholar 

  32. Xue, W., Li, T.: Aspect based sentiment analysis with gated convolutional networks. In: ACL, pp. 2514–2523 (2018)

    Google Scholar 

  33. Yang, M., Tu, W., Wang, J., Xu, F., Chen, X.: Attention based LSTM for target dependent sentiment classification. In: AAAI, pp. 5013–5014 (2017)

    Google Scholar 

  34. Yang, Z., Yang, D., Dyer, C., He, X., Smola, A.J., Hovy, E.H.: Hierarchical attention networks for document classification. In: NAACL HLT, pp. 1480–1489 (2016)

    Google Scholar 

  35. Yuan, Z., Wu, F., Liu, J., Wu, C., Huang, Y., Xie, X.: Neural review rating prediction with user and product memory. In: CIKM, pp. 2341–2344 (2019)

    Google Scholar 

  36. Zhang, L., Wang, S., Liu, B.: Deep learning for sentiment analysis: A survey. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 8(4), e1253 (2018)

    Google Scholar 

  37. Zhang, Y., Wu, J., Wang, J., Xing, C.: A transformation-based framework for KNN set similarity search. IEEE Trans. Knowl. Data Eng. 32(3), 409–423 (2020)

    Article  Google Scholar 

  38. Zhao, K., et al.: Modeling patient visit using electronic medical records for cost profile estimation. In: DASFAA, pp. 20–36 (2018)

    Google Scholar 

  39. Zhao, K., et al.: Discovering subsequence patterns for next POI recommendation. In: IJCAI, pp. 3216–3222 (2020)

    Google Scholar 

  40. Zou, Y., Gui, T., Zhang, Q., Huang, X.: A lexicon-based supervised attention model for neural sentiment analysis. In: COLING, pp. 868–877 (2018)

    Google Scholar 

Download references

Acknowledgment

This work was supported by National Key R&D Program of China: 2018YFB1404401, 2018YFB1402701 and 2020AAA0109603; State Key Laboratory of Computer Architecture (ICT,CAS) under Grant No. CARCHA202008 and Institute of Precision Medicine, Tsinghua University.

Author information

Authors and Affiliations

  1. DCST, BNRist, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China

    Bing Tian

  2. BNRist, Department of Computer Science and Technology, RIIT, Institute of Internet Industry, Tsinghua University, Beijing, China

    Yong Zhang & Chunxiao Xing

Authors
  1. Bing Tian
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yong Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Chunxiao Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yong Zhang .

Editor information

Editors and Affiliations

  1. University of Macau, Macau, China

    Leong Hou U

  2. University of Caen Normandie, Caen, France

    Marc Spaniol

  3. Osaka University, Osaka, Japan

    Yasushi Sakurai

  4. South China University of Technology, Guangzhou, China

    Junying Chen

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Tian, B., Zhang, Y., Xing, C. (2021). Improving Document-Level Sentiment Classification with User-Product Gated Network. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12858. Springer, Cham. https://doi.org/10.1007/978-3-030-85896-4_31

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-85896-4_31

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-85895-7

  • Online ISBN: 978-3-030-85896-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics