Skip to main content
SpringerLink
Log in

Semantic and syntactic analysis in learning representation based on a sentiment analysis model

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The rapid development of e-commerce gives researchers confidence that customers will be willing to share more and more online data, which in turn, would allow for improved mining algorithms. Many companies also foresee vast profits in mining data from online interaction, behavior, and activity. Opinion mining, also known as sentiment analysis, means automatically detecting and understanding personal expressions about a product or service from customer textual reviews. Recently, aspect-based sentiment analysis has become widely interesting to researchers, particularly with respect to embedded words. Algorithms such as word2vec and GloVe perform well when it comes to capturing analogies and toward lexical semantics in general. However, more complex algorithms are needed to address this issue more precisely, using larger corpora and special kinds of data. This paper introduces a knowledge representation approach that centers on aspect rating and weighting. The study focuses on how to understand the nature of sentimental representation using a multilayer architecture. We present a model that uses a mixture of semantic and syntactic components to capture both semantic and sentimental information. This model shares its probability foundation with the words recognized by word2vec and builds on our prior work concerning opinion-aspect relation analysis. This new algorithm is designed specifically, however, to discover sentiment-enriched embedding rather than word similarities. Experiments were performed using a review dataset from the electronic domain. Results show that the model achieved both appropriate levels of detail and rich representation capabilities.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. http://nlp.stanford.edu/software/stanford-dependencies.shtml

  2. https://stanfordnlp.github.io/CoreNLP/coref.html

  3. https://nlp.stanford.edu/software/CRF-NER.shtml

  4. https://www.cs.uic.edu/liub/FBS/sentiment-analysis.html

  5. http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools

  6. http://nlp.stanford.edu/software/stanford-dependencies.shtml

  7. https://wordnet.princeton.edu/

References

  1. Salton G, Singhal A, Buckley C, Mitra M (1996) Automatic text decomposition using text segments and text themes. In: Proceedings of the the 7th ACM Conference on Hypertext, HYPERTEXT ’96. ACM, New York, pp 53–65

  2. Goldstein J, Kantrowitz M, Mittal V, Carbonell J (1999) Summarizing text documents: Sentence selection and evaluation metrics. In: Proceedings of the 22nd annual international ACM SIGIR conference on research and development in information retrieval, SIGIR ’99. ACM, New York, pp 121–128

  3. Gross O, Doucet A, Toivonen H (2014) Document summarization based on word associations. In: Proceedings of the 37th international ACM SIGIR conference on research & development in information retrieval, SIGIR ’14. ACM, New York, pp 1023–1026

  4. Turney PD (2002) Thumbs up or thumbs down?: Semantic orientation applied to unsupervised classification of reviews. In: Proceedings of the 40th annual meeting on association for computational linguistics, ACL ’02. Association for Computational Linguistics, Stroudsburg, pp 417–424

  5. Pang B, Lee L, Vaithyanathan S (2002) Thumbs up?: Sentiment classification using machine learning techniques. In: Proceedings of the ACL-02 conference on empirical methods in natural language processing - vol. 10, EMNLP ’02. Association for Computational Linguistics, Stroudsburg, pp 79–86

  6. Baccianella S, Esuli A, Sebastiani F (2010) Sentiwordnet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining. In: Proceedings of the 7th conference on international language resources and evaluation (LREC’10), European languages resources association (ELRA)

  7. Vo A-D, Ock C-Y (2012) Sentiment classification: A combination of pmi, sentiwordnet and fuzzy function. In: Computational collective intelligence. Technologies and applications. Springer, Berlin, pp 373–382

    Chapter  Google Scholar 

  8. Su Y-J, Wu H-T, Chen Y-Q, Hu W-C (2018) Using cclm to promote the accuracy of intelligent sentiment analysis classifier for chinese social media service. J Netw Intell 3(2):113–125

    Google Scholar 

  9. Riloff E, Wiebe J (2003) Learning extraction patterns for subjective expressions. In: Proceedings of the 2003 conference on empirical methods in natural language processing, EMNLP ’03. Association for Computational Linguistics, Stroudsburg, pp 105–112

  10. Yu H, Hatzivassiloglou V (2003) Towards answering opinion questions: Separating facts from opinions and identifying the polarity of opinion sentences. In: Proceedings of the 2003 conference on empirical methods in natural language processing, EMNLP ’03. Association for Computational Linguistics, Stroudsburg, pp 129–136

  11. Mukund S, Srihari R (2010) A vector space model for subjectivity classification in urdu aided by co-training. In: Proceedings of the 23rd International Conference on Computational Linguistics: Posters, COLING ’10. Association for Computational Linguistics, Stroudsburg, pp 860–868

  12. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the 10th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’04. ACM, New York, pp 168–177

  13. Hu M, Liu B (2004) Mining opinion features in customer reviews. In: Proceedings of the 19th national conference on artifical intelligence, AAAI’04, pp 755–760. AAAI Press

  14. Popescu A-M, Etzioni O (2005) Extracting product features and opinions from reviews. In: Proceedings of the conference on human language technology and empirical methods in natural language processing, HLT ’05. Association for Computational Linguistics, Stroudsburg, pp 339–346

  15. Moghaddam S, Ester M (2011) Ilda: Interdependent lda model for learning latent aspects and their ratings from online product reviews. In: Proceedings of the 34th international ACM SIGIR conference on research and development in information retrieval, SIGIR ’11. ACM, New York, pp 665–674

  16. Long C, Zhang J, Zhut X (2010) A review selection approach for accurate feature rating estimation. In: Proceedings of the 23rd international conference on computational linguistics: Posters, COLING ’10. Association for Computational Linguistics, Stroudsburg, pp 766–774

  17. Hofmann T (2017) Probabilistic latent semantic indexing. SIGIR Forum 51:211–218

    Article  Google Scholar 

  18. Blei D M, Ng A Y, Jordan M I (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022

    MATH  Google Scholar 

  19. Titov I, McDonald R (2008) Modeling online reviews with multi-grain topic models. In: Proceedings of the 17th international conference on world wide web, WWW ’08. ACM, New York, pp 111–120

  20. Brody S, Elhadad N (2010) An unsupervised aspect-sentiment model for online reviews. In: Human Language technologies: The 2010 annual conference of the North American chapter of the association for computational linguistics, HLT ’10. Association for Computational Linguistics, Stroudsburg, pp 804–812

  21. Zhuang L, Jing F, Zhu X-Y (2006) Movie review mining and summarization. In: Proceedings of the 15th ACM international conference on information and knowledge management, CIKM ’06. ACM, New York, pp 43–50

  22. Somasundaran S, Ruppenhofer J, Wiebe J (2008) Discourse level opinion relations: An annotation study. In: Proceedings of the 9th SIGdial workshop on discourse and dialogue, SIGdial ’08. Association for Computational Linguistics, Stroudsburg, pp 129–137

  23. Kobayashi N, Iida R, Inui K, Matsumoto Y (2006) Opinion mining on the web by extracting subject-aspect-evaluation relations. In: AAAI spring symposium: Computational approaches to analyzing Weblogs, pp 86–91

  24. Qiu G, Liu B, Bu J, Chen C (2009) Expanding domain sentiment lexicon through double propagation. In: Proceedings of the 21st international jont conference on artifical intelligence, IJCAI’09. Morgan Kaufmann Publishers Inc., San Francisco, pp 1199–1204

  25. Zhang L, Liu B, Lim SH, O’Brien-Strain E (2010) Extracting and ranking product features in opinion documents. In: Proceedings of the 23rd international conference on computational linguistics: Posters COLING ’10. Association for Computational Linguistics, Stroudsburg, pp 1462–1470

  26. Vo A, Nguyen Q, Ock C (2018) Opinion–aspect relations in cognizing customer feelings via reviews. IEEE Access 6:5415–5426

    Article  Google Scholar 

  27. Zhu P, Qian T (2018) Enhanced aspect level sentiment classification with auxiliary memory. Association for Computational Linguistics, Santa Fe, pp 1077–1087

    Google Scholar 

  28. He R, Lee W S, Ng H T, Dahlmeier D (2018) Exploiting document knowledge for aspect-level sentiment classification. arXiv:1806.04346

  29. Shu L, Xu H, Liu B (2017) Lifelong learning crf for supervised aspect extraction. In: Proceedings of the 55th annual meeting of the association for computational linguistics (vol 2: Short Papers), pp 148–154. Association for Computational Linguistics

  30. Bi J, Zhang C (2018) An empirical comparison on state-of-the-art multi-class imbalance learning algorithms and a new diversified ensemble learning scheme. Knowl-Based Syst 158:81–93

    Article  Google Scholar 

  31. Yu J, Zha Z-J, Wang M, Chua T-S (2011) Aspect ranking: Identifying important product aspects from online consumer reviews. In: Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies - vol 1, HLT ’11. Association for Computational Linguistics, Stroudsburg, pp 1496–1505

  32. Manevitz L M, Yousef M (2002) One-class svms for document classification. J Mach Learn Res 2:139–154

    MATH  Google Scholar 

  33. Liu B, Hu M, Cheng J (2005) Opinion observer: Analyzing and comparing opinions on the web. In: Proceedings of the 14th international conference on world wide web, WWW ’05. ACM, New York, pp 342–351

  34. Ghani R, Probst K, Liu Y, Krema M, Fano A (2006) Text mining for product attribute extraction. SIGKDD Explor Newsl 8:41–48

    Article  Google Scholar 

  35. Kovelamudi S, Ramalingam S, Sood A, Varma V (2011) Domain independent model for product attribute extraction from user reviews using wikipedia. In: Proceedings of 5th international joint conference on natural language processing, pp 1408–1412. Asian Federation of Natural Language Processing

  36. Toh Z, Su J (2016) Nlangp at semeval-2016 task 5: Improving aspect based sentiment analysis using neural network features. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pp 282–288. Association for Computational Linguistics

  37. Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, AL-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, Hoste V, Apidianaki M, Tannier X, Loukachevitch N, Kotelnikov E, Bel N, Jiménez-Zafra SM, Eryiġit G (2016) Semeval-2016 task 5: Aspect based sentiment analysis. In: Proceedings of the 10th international workshop on semantic evaluation (SemEval-2016), pp 19–30. Association for Computational Linguistics

  38. Xenos D, Theodorakakos P, Pavlopoulos J, Malakasiotis P, Androutsopoulos I (2016) Aueb-absa at semeval-2016 task 5: Ensembles of classifiers and embeddings for aspect based sentiment analysis. In: Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pp 312–317. Association for Computational Linguistics

  39. Turney P D, Pantel P (2010) From frequency to meaning: Vector space models of semantics. J Artif Int Res 37:141–188

    MathSciNet  MATH  Google Scholar 

  40. Bengio Y, Ducharme R, Vincent P, Janvin C (2003) A neural probabilistic language model. J Mach Learn Res 3:1137–1155

    MATH  Google Scholar 

  41. Mikolov T, Sutskever I, Chen K, Corrado G, Dean J (2013) Distributed representations of words and phrases and their compositionality. In: Proceedings of the 26th international conference on neural information processing systems - vol. 2, NIPS’13. Curran Associates Inc., USA, pp 3111–3119

  42. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. arXiv:1301.3781

  43. Pelevina M, Arefiev N, Biemann C, Panchenko A (2016) Making sense of word embeddings, In: Proceedings of the 1st workshop on representation learning for NLP, pp 174–183. Association for Computational Linguistics

  44. Kwon DOS, Kim K, Ko Y (2018) Word sense disambiguation based on word similarity calculation using word vector representation from a knowledge-based graph. In: Proceedings of the 27th international conference on computational linguistics, pp 2704–2714. Association for Computational Linguistics

  45. Trask A, Michalak P, Liu J (2015) Fast and accurate method for word sense disambiguation in neural word embeddings. arXiv:1511.06388

  46. Wang R, Zhao H, Ploux S, Lu B-L, Utiyama M, Sumita E (2018) Graph-based bilingual word embedding for statistical machine translation. ACM Trans Asian Low-Resour Lang Inf Process 17:31,1–31,23

    Google Scholar 

  47. Nguyen Q, Vo A, Shin J, Ock C (2018) Effect of word sense disambiguation on neural machine translation: A case study in korean. IEEE Access 6:38512–38523

    Article  Google Scholar 

  48. Das A, Ganguly D, Garain U (2017) Named entity recognition with word embeddings and wikipedia categories for a low-resource language. ACM Trans Asian Low-Resour Lang Inf Process 6:18:1–18:19

    Google Scholar 

  49. Amer N O, Mulhem P, Gery M (2016) Toward word embedding for personalized information retrieval. arXiv:1606.06991

  50. Kuzi S, Shtok A, Kurland O (2016) Query expansion using word embeddings. In: Proceedings of the 25th ACM international on conference on information and knowledge management, CIKM ’16. ACM, New York, pp 1929–1932

  51. Kenter T, de Rijke M (2015) Short text similarity with word embeddings. In: Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, CIKM ’15. ACM, New York, pp 1411–1420

  52. Song Y, Lee C-J (2017) Embedding projection for query understanding. In: Proceedings of the 26th international conference on world wide web companion, WWW ’17 Companion. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, pp 839–840

  53. Nalisnick E, Mitra B, Craswell N, Caruana R (2016) Improving document ranking with dual word embeddings. In: Proceedings of the 25th international conference companion on world wide web, WWW ’16 Companion. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, pp 83–84

  54. Ganguly D, Roy D, Mitra M, Jones GJ (2015) Word embedding based generalized language model for information retrieval. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’15. ACM, New York, pp 795–798

  55. Guo J, Fan Y, Ai Q, Croft WB (2016) Semantic matching by non-linear word transportation for information retrieval. In: Proceedings of the 25th ACM international on conference on information and knowledge management, CIKM ’16. ACM, New York, pp 701–710

  56. Balikas G, Amini M (2016) An empirical study on large scale text classification with skip-gram embeddings. arXiv:1606.06623

  57. Wang S, Tang J, Aggarwal C, Liu H (2016) Linked document embedding for classification. In: Proceedings of the 25th ACM international on conference on information and knowledge management, CIKM ’16. ACM, New York, pp 115– 124

  58. Liang S, Zhang X, Ren Z, Kanoulas E (2018) Dynamic embeddings for user profiling in twitter. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & Data mining, KDD ’18. ACM, New York, pp 1764–1773

  59. Liu B (2012) Sentiment analysis and opinion mining. Morgan & Claypool Publishers

  60. Pang B, Lee L (2008) Opinion mining and sentiment analysis. Found Trends Inf Retr 2:1–135

    Article  Google Scholar 

  61. Mitchell J, Lapata M (2008) Vector-based models of semantic composition. In: Proceedings of ACL-08:, HLT, pp 236–244

  62. Hermann K M, Blunsom P (2013) The role of syntax in vector space models of compositional semantics. In: Proceedings of the 51st annual meeting of the association for computational linguistics

  63. Hermann K M, Blunsom P (2014) Multilingual models for compositional distributed semantics. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (vol. 1: Long Papers), pp 58–68. Association for Computational Linguistics

  64. McAuley J, Yang A (2016) Addressing complex and subjective product-related queries with customer reviews. In: World Wide Web

  65. Wang H, Lu Y, Zhai C (2010) Latent aspect rating analysis on review text data: A rating regression approach. In: Proceedings of the 16th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’10. ACM, New York, pp 783–792

  66. Wang H, Lu Y, Zhai C (2011) Latent aspect rating analysis without aspect keyword supervision. In: Proceedings of the 17th ACM SIGKDD international conference on knowledge discovery and data mining, KDD ’11. ACM, New York, pp 618–626

  67. Tsytsarau M, Palpanas T (2012) Survey on mining subjective data on the web. Data Min Knowl Disc 24:478–514

    Article  Google Scholar 

  68. Tang H, Tan S, Cheng X (2009) A survey on sentiment detection of reviews. Expert Syst Appl 36:10760–10773

    Article  Google Scholar 

  69. Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis. Know-Based Syst 89:14–46

    Article  Google Scholar 

  70. Medhat W, Hassan A, Korashy H (2014) Sentiment analysis algorithms and applications: A survey. Ain Shams Eng J 5(4):1093–1113

    Article  Google Scholar 

  71. Vo A-D, Nguyen Q-P, Ock C-Y (2018) Automatic knowledge extraction for aspect-based sentiment analysis of customer reviews. In: Proceedings of the 10th international conference on computer modeling and simulation, ICCMS 2018. ACM, New York, pp 110–113

  72. Wang Y, Huang M, Zhu X, Zhao L (2016) Attention-based LSTM for aspect-level sentiment classification. In: Proceedings of the 2016 conference on empirical methods in natural language processing, (Austin, Texas), pp 606–615, Association for Computational Linguistics

  73. Chen P, Sun Z, Bing L, Yang W (2017) Recurrent attention network on memory for aspect sentiment analysis. In: Proceedings of the 2017 conference on empirical methods in natural language processing, (Copenhagen, Denmark), pp 452–461, Association for Computational Linguistics

Download references

Acknowledgements

This work was supported by the ICT R&D Program of MSIP/IITP (2013-0-00179, Development of Core Technology for Context-aware Deep-Symbolic Hybrid Learning and Construction of Language Resources).

Author information

Authors and Affiliations

  1. Department of IT Convergence, University of Ulsan, Ulsan, 44610, Republic of Korea

    Anh-Dung Vo, Quang-Phuoc Nguyen & Cheol-Young Ock

Authors
  1. Anh-Dung Vo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Quang-Phuoc Nguyen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Cheol-Young Ock
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cheol-Young Ock.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Vo, AD., Nguyen, QP. & Ock, CY. Semantic and syntactic analysis in learning representation based on a sentiment analysis model. Appl Intell 50, 663–680 (2020). https://doi.org/10.1007/s10489-019-01540-2

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01540-2

Keywords

Search

Navigation