Skip to main content
SpringerLink
Log in

A mixed unsupervised method for aspect extraction using BERT

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

With the increase of unstructured text on social media platforms from user opinions, deep neural network techniques have significantly contributed to the aspect extraction subtask of Aspect-Based Sentiment Analysis (ABSA). In a multi-sentence review, sentences are contextually interdependent, and static word embedding generates similar representations for the same word in different domains. Hence, existing techniques cannot capture inter-sentence dependencies for valid multi-word aspect extraction. Further, incorporating conceptual information to associate the context and aspect terms is still a challenging task. Therefore, this paper aims to remove inadequate information and capture aspect co-referencing by adding a sentence coreference resolution step before performing ABSA in an unsupervised rule-based method. Next, domain irrelevant aspects are pruned out using contextual embedding. Furthermore, aspects extracted using unsupervised way are given as labeled in training the hierarchical attention-based network using pre-trained language model BERT, Bidirectional Encoder Representations from Transformers. The experimental results on the SemEval-16 dataset show that F-score results are between 2.5% and 5% better than recent supervised deep learning approaches for laptop and restaurant domains, respectively.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. http://alt.qcri.org/semeval2016/task5/index.php?id=data-and-tools

  2. https://nlp.stanford.edu/software/lex-parser.shtml

References

  1. Akhtar S, Garg T, Ekbal A (2020) Neurocomputing multi-task learning for aspect term extraction and aspect sentiment classification. Neurocomputing 398:247–256. https://doi.org/10.1016/j.neucom.2020.02.093

  2. Ba JL, Kiros JR, Hinton GE (2016) Layer normalization

  3. Blair-Goldensohn S, Hannan K, McDonald R et al (2008) Building a sentiment summarizer for local service reviews. WWW Work NLP Inf Explos Era:339–348

  4. Chauhan GS, Kumar Meena Y (2018) Prominent aspect term extraction in aspect based sentiment analysis. In: 3rd international conference and workshops on recent advances and innovations in engineering, ICRAIE 2018. Institute of Electrical and Electronics Engineers Inc, pp 1–6. https://doi.org/10.1109/ICRAIE.2018.8710408

  5. Chauhan GS, Meena YK (2019) YouTube video ranking by aspect-based sentiment analysis on user feedback. In: Advances in intelligent systems and computing. Springer Verlag, pp 63–71, vol 900. Springer, Singapore. https://doi.org/10.1007/978-981-13-3600-3_6

  6. Chauhan GS, Agrawal P, Meena YK (2019) Aspect-based sentiment analysis of students’ feedback to improve teaching–learning process. In: Smart innovation, Systems and Technologies. Springer Science and Business Media Deutschland GmbH, pp 259–266, vol 107. Springer, Singapore. https://doi.org/10.1007/978-981-13-1747-7_25

  7. Chen T, Xu R, He Y, Wang X (2017) Improving sentiment analysis via sentence type classification using BiLSTM-CRF and CNN. Expert Syst Appl 72:221–230. https://doi.org/10.1016/j.eswa.2016.10.065

    Article  Google Scholar 

  8. Devlin J, Chang MW, Lee K, Toutanova K (2019) BERT: Pre-training of deep bidirectional transformers for language understanding. NAACL HLT 2019–2019 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol - Proc Conf 1:4171–4186

    Google Scholar 

  9. Dilawar N, Majeed H Sentence Vector Representation Methods for Aspect Category Detection. 1–10

  10. Do HH, Prasad PWC, Maag A, Alsadoon A (2019) Deep learning for aspect-based sentiment analysis: a comparative review. Expert Syst Appl 118:272–299. https://doi.org/10.1016/j.eswa.2018.10.003

    Article  Google Scholar 

  11. Fan GF, Yu M, Dong SQ, Yeh YH, Hong WC (2021) Forecasting short-term electricity load using hybrid support vector regression with grey catastrophe and random forest modeling. Util Policy 73:101294. https://doi.org/10.1016/j.jup.2021.101294

    Article  Google Scholar 

  12. Hoang M, Rouces J (2019) Aspect-based sentiment analysis using BERT

  13. Hu M, Liu B (2004) Mining and summarizing customer reviews. In: KDD-2004 - proceedings of the tenth ACM SIGKDD international conference on knowledge discovery and data mining. ACM Press, New York, pp 168–177

    Chapter  Google Scholar 

  14. Jebbara S, Cimiano P (2016) Aspect-based sentiment analysis using a two-step neural network architecture. In: Communications in computer and information science. Springer Verlag, 641, pp 153–167. https://doi.org/10.1007/s10994-013-5413-0

  15. Kang Y, Zhou L (2017) RubE: rule-based methods for extracting product features from online consumer reviews. Inf Manag 54:166–176. https://doi.org/10.1016/j.im.2016.05.007

    Article  Google Scholar 

  16. Kersting JGM (2021) Human language comprehension in aspect phrase extraction with importance weighting. Nat Lang Process Inf Syst 12801:231–242. https://doi.org/10.1007/978-3-030-80599-9_21

  17. Li X, Lam W (2017) Deep multi-task learning for aspect term extraction with memory interaction, EMNLP. Association for Computational Linguistics, pp 2886–2892. https://doi.org/10.18653/v1/D17-1310

  18. Li S, Zhou L, Li Y (2015) Improving aspect extraction by augmenting a frequency-based method with web-based similarity measures. Inf Process Manag 51(1):58–67. https://doi.org/10.1016/j.ipm.2014.08.005

  19. Li X, Bing L, Zhang W, Lam W (2019) Exploiting BERT for end-to-end aspect-based sentiment analysis. 34–41. https://doi.org/10.18653/v1/d19-5505

  20. Liu B, Hsu W, Ma Y (1998) Integrating classification and association rule mining

  21. Liu P, Joty S, Meng H (2015) Fine-grained opinion mining with recurrent neural networks and word Embeddings. Association for Computational Linguistics

  22. Liu MZ, Zhou FY, Chen K, Zhao Y (2021) Co-attention networks based on aspect and context for aspect-level sentiment analysis. Knowledge-Based Syst 217:106810. https://doi.org/10.1016/j.knosys.2021.106810

  23. Ma D, Li S, Zhang X, Wang H (2017) Interactive attention networks for aspect-level sentiment classification, IJCAI, Artificial Intelligence (cs.AI); Computation and Language (cs.CL). https://doi.org/10.48550/arXiv.1709.00893

  24. Meškelė D, Frasincar F (2020) ALDONAr: a hybrid solution for sentence-level aspect-based sentiment analysis using a lexicalized domain ontology and a regularized neural attention model. Inf Process Manag 57(3):102211. https://doi.org/10.1016/j.ipm.2020.102211

  25. Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. In: 1st international conference on learning representations, ICLR 2013 - workshop track proceedings. International conference on learning representations, ICLR

  26. Ozyurt B, Akcayol MA (2021) A new topic modeling based approach for aspect extraction in aspect based sentiment analysis: SS-LDA. Expert Syst Appl 168:114231. https://doi.org/10.1016/j.eswa.2020.114231

  27. Peng C, Zhongqian S, Lidong B, Yang W (n.d.) Recurrent Attention Network on Memory for Aspect Sentiment Analysis

  28. Peters ME, Neumann M, Iyyer M et al (2018) Improving language understanding by. OpenAI:1–10

  29. Pontiki M, Galanis D, Papageorgiou H, et al (n.d.) SemEval-2016 Task 5: Aspect Based Sentiment Analysis

  30. Popescu AM, Etzioni O (2005) Extracting product features and opinions from reviews. In: HLT/EMNLP 2005 - human language technology conference and conference on empirical methods in natural language processing. Proceedings of the Conference. Association for Computational Linguistics, Morristown, pp 339–346. https://aclanthology.org/H05-1043

  31. Poria S, Cambria E, Gelbukh A (2016) Aspect extraction for opinion mining with a deep convolutional neural network. Knowledge-Based Syst 108:42–49. https://doi.org/10.1016/j.knosys.2016.06.009

    Article  Google Scholar 

  32. Qiu G, Liu B, Bu J, Chen C (2011) Opinion word expansion and target extraction through double propagation

  33. Radford A, Wu J, Child R, et al (2018) Language models are unsupervised multitask learners

  34. Rajput R, Solanki A (2016) International journal of computer science and Mobile computing review of sentimental analysis methods using lexicon based approach. Int J Comput Sci Mob Comput 5:159–166

    Google Scholar 

  35. Rajput R, Solanki A (2017) Real time sentiment analysis of tweets using machine learning and semantic analysis. Commun Comput Syst - Proc Int Conf Commun Comput Syst ICCCS 2016:687–692. https://doi.org/10.1201/9781315364094-123

  36. Rana TA, Cheah YN (2016) Aspect extraction in sentiment analysis: comparative analysis and survey. Artif Intell Rev 46:459–483. https://doi.org/10.1007/s10462-016-9472-z

    Article  Google Scholar 

  37. Rana TA, Cheah YN (2017) A two-fold rule-based model for aspect extraction. Expert Syst Appl 89:273–285. https://doi.org/10.1016/j.eswa.2017.07.047

  38. Rana TA, Cheah YN, Letchmunan S (2016) Topic modeling in sentiment analysis: a systematic review. J ICT Res Appl 10:76–93

    Article  Google Scholar 

  39. Ravi K, Ravi V (2015) A survey on opinion mining and sentiment analysis: tasks, approaches and applications. Knowledge-Based Syst 89:14–46. https://doi.org/10.1016/j.knosys.2015.06.015

  40. Ren Z, Shen Q, Diao X, Xu H (2021) A sentiment-aware deep learning approach for personality detection from text. Inf Process Manag 58:102532. https://doi.org/10.1016/j.ipm.2021.102532

  41. Rojas-Barahona LM (2016) Deep learning for sentiment analysis. Lang Linguist Compass 10:701–719. https://doi.org/10.1111/lnc3.12228

    Article  Google Scholar 

  42. Ruder S, Ghaffari P, Breslin JG (n.d.) A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis

  43. Samha AK, Li Y, Zhang J (2015) Aspect-based opinion mining from product reviews using conditional random fields. Undefined

  44. Schouten K, Frasincar F (2016) Survey on aspect-level sentiment analysis. IEEE Trans Knowl Data Eng 28:813–830. https://doi.org/10.1109/TKDE.2015.2485209

    Article  Google Scholar 

  45. Schouten K, van der Weijde O, Frasincar F, Dekker R (2017) Supervised and unsupervised aspect category detection for sentiment analysis with co-occurrence data. IEEE Trans Cybern 48:1263–1275. https://doi.org/10.1109/TCYB.2017.2688801

    Article  Google Scholar 

  46. Singh Chauhan G, Kumar Meena Y, Gopalani D, Nahta R (2020) A two-step hybrid unsupervised model with attention mechanism for aspect extraction Expert Syst Appl 113673. https://doi.org/10.1016/j.eswa.2020.113673

  47. Singh Chauhan G, Kumar Meena Y, Gopalani D, Nahta R (2020) A two-step hybrid unsupervised model with attention mechanism for aspect extraction. Expert Syst Appl 161:113673. https://doi.org/10.1016/j.eswa.2020.113673

    Article  Google Scholar 

  48. Singh T, Nayyar A, Solanki A (2020) Multilingual opinion mining movie recommendation system using RNN. Springer, Singapore

    Book  Google Scholar 

  49. Srividya K, Mary Sowjanya A (2021) NA-DLSTM – a neural attention based model for context aware aspect-based sentiment analysis. Mater Today Proc https://doi.org/10.1016/j.matpr.2021.01.782

  50. Su J, Tang J, Jiang H, Lu Z, Ge Y, Song L, Xiong D, Sun L, Luo J (2021) Enhanced aspect-based sentiment analysis models with progressive self-supervised attention learning. Artif Intell 296:103477. https://doi.org/10.1016/j.artint.2021.103477

    Article  MathSciNet  MATH  Google Scholar 

  51. Sun C, Huang L, Qiu X (2019) Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. NAACL HLT 2019–2019 Conf North Am Chapter Assoc Comput Linguist Hum Lang Technol - Proc Conf 1:380–385

    Google Scholar 

  52. Tan X, Cai Y, Xu J, Leung HF, Chen W, Li Q (2020) Improving aspect-based sentiment analysis via aligning aspect embedding. Neurocomputing 383:336–347. https://doi.org/10.1016/j.neucom.2019.12.035

    Article  Google Scholar 

  53. Tay Y, Anh Tuan L, Cheung Hui S (2017) Dyadic memory networks for aspect-based sentiment analysis. https://doi.org/10.1145/3132847.3132936

  54. Toh Z, Su J (2016) NLANGP at SemEval-2016 task 5: improving aspect based sentiment analysis using neural network features

  55. Vaswani A, Shazeer N, Parmar N, et al (2017) Attention is all you need. Adv Neural Inf Process Syst 2017-Decem:5999–6009

  56. Wang X, Li F, Zhang Z, Xu G, Zhang J, Sun X (2021) A unified position-aware convolutional neural network for aspect based sentiment analysis. Neurocomputing. 450:91–103. https://doi.org/10.1016/j.neucom.2021.03.092

    Article  Google Scholar 

  57. Wang X, Xu G, Zhang Z, Jin L, Sun X (2021) End-to-end aspect-based sentiment analysis with hierarchical multi-task learning. Neurocomputing. 455:178–188. https://doi.org/10.1016/j.neucom.2021.03.100

    Article  Google Scholar 

  58. Wu C, Wu F, Wu S, Yuan Z, Huang Y (2018) A hybrid unsupervised method for aspect term and opinion target extraction. Knowledge-Based Syst 148:66–73. https://doi.org/10.1016/j.knosys.2018.01.019

  59. Wu C, Xiong Q, Yang Z, Gao M, Li Q, Yu Y, Wang K, Zhu Q (2021) Residual attention and other aspects module for aspect-based sentiment analysis. Neurocomputing 435:42–52. https://doi.org/10.1016/j.neucom.2021.01.019

  60. Yang B, Cardie C (2012) Extracting opinion expressions with semi-Markov conditional random fields. Association for Computational Linguistics

  61. Yang C, Zhang H, Jiang B, Li K (2019) Aspect-based sentiment analysis with alternating coattention networks. Inf Process Manag 56:463–478. https://doi.org/10.1016/j.ipm.2018.12.004

  62. Yuan Z, Wu S, Wu F, Liu J, Huang Y (2018) Domain attention model for multi-domain sentiment classification. Knowledge-Based Syst 155:1–10. https://doi.org/10.1016/j.knosys.2018.05.004

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science & Engineering, Malaviya National Institute of Technology, Jaipur, Rajasthan, India

    Ganpat Singh Chauhan, Yogesh Kumar Meena, Dinesh Gopalani & Ravi Nahta

Authors
  1. Ganpat Singh Chauhan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yogesh Kumar Meena
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dinesh Gopalani
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Ravi Nahta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ganpat Singh Chauhan.

Ethics declarations

Conflict of interest

The authors confirm no conflict of interest with the submission and ensure that the manuscript has not been published nor under consideration else.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Chauhan, G.S., Meena, Y.K., Gopalani, D. et al. A mixed unsupervised method for aspect extraction using BERT. Multimed Tools Appl 81, 31881–31906 (2022). https://doi.org/10.1007/s11042-022-13023-7

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-13023-7

Keywords

Search

Navigation