Abstract
Aspect category detection is an essential task in aspect-based sentiment analysis. Most previous works use labeled data and apply a supervised learning approach to detect the aspect category. However, to avoid the dependency on labeled data, some researchers have also applied unsupervised learning approaches, wherein variants of topic models and neural network-based models have been built for this task. These unsupervised methods focus on co-occurrences of words and ignore the contextual meaning of the words in the given sentence. Thus, such models perform reasonably well in detecting the explicitly expressed aspect category but often fail in identifying the implicitly expressed aspect category in the sentence. This paper focuses on the contextual meaning of the word. It adopts a document clustering approach requiring minimal user guidance, i.e., only a small set of seed words for each aspect category to detect efficiently implicit and explicit aspect categories. A novel BERT-based Attentive Representation Learning with Adversarial Training (BARLAT) model is presented in this paper, which utilizes domain-based contextual word embedding (BERT) for generating the sentence representation and uses these representations for clustering the sentences through attentive representation learning. Further, the model parameters are generalized better by performing adversarial training, which adds perturbations to the cluster representations. BARLAT is the first nearly unsupervised method that uses the contextual meaning of the words for learning the aspect categories through an adversarial attentive learning approach. The performance of BARLAT is compared with various state-of-the-art models using F1-score on Laptop and Restaurant datasets. The experimental results show that BARLAT outperforms the best existing model by a margin of 1.1% and 2.3% on Restaurant and Laptop datasets, respectively.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Hu M, Liu B (2004) Mining and summarizing customer reviews. In: Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp 168–177
Liu B, Zhang L (2012) A survey of opinion mining and sentiment analysis. In: Mining text data. Springer, pp 415–463
Pontiki M, Galanis D, Papageorgiou H, Androutsopoulos I, Manandhar S, Al-Smadi M, Al-Ayyoub M, Zhao Y, Qin B, De Clercq O, et al. (2016)Semeval-2016 task 5: Aspect based sentiment analysis. In: 10th international workshop on semantic evaluation (SemEval 2016)
Qiu G, Liu B, Jiajun B, Chen C (2011) Opinion word expansion and target extraction through double propagation. Comput Linguist 37(1):9–27
Blei DM, Ng AY, Jordan MI (2003) Latent dirichlet allocation. J Mach Learn Res 3:993–1022
Brody S, Elhadad N (201) An unsupervised aspect-sentiment model for online reviews. In: Human language technologies: the 2010 annual conference of the North American chapter of the association for computational linguistics. Association for Computational Linguistics, pp 804–812
Zhao WX, Jiang J, Yan H, Li X (2010) Jointly modeling aspects and opinions with a maxent-lda hybrid. In: Proceedings of the 2010 conference on empirical methods in natural language processing. Association for Computational Linguistics, pp 56–65
Chen Z, Mukherjee A, Liu B (2014) Aspect extraction with automated prior knowledge learning. In: Proceedings of the 52nd annual meeting of the association for computational linguistics (volume 1: long papers), pp 347–358
Mimno D, Wallach H, Talley E, Leenders M, McCallum A (2011) Optimizing semantic coherence in topic models. In: Proceedings of the 2011 conference on empirical methods in natural language processing, pp 262–272
Miao Y, Yu L, Blunsom P (2016) Neural variational inference for text processing. In: International conference on machine learning, pp 1727–1736
Srivastava A, Sutton C (2017) Autoencoding variational inference for topic models. Preprint arXiv:1703.01488
Liao M, Li J, Zhang H, Wang L, Wu X, Wong K-F (2019) Coupling global and local context for unsupervised aspect extraction. In: Proceedings of the 2019 conference on empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP-IJCNLP), pp 4571–4581
He R, Lee WS, Ng HT, Dahlmeier D (2017) An unsupervised neural attention model for aspect extraction. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 1: long papers), pp 388–397
Tulkens S, van Cranenburgh A (2020) Embarrassingly simple unsupervised aspect extraction. Preprint arXiv:2004.13580
Mikolov T, Chen K, Corrado G, Dean J (2013) Efficient estimation of word representations in vector space. Preprint arXiv:1301.3781
Devlin J, Chang M-W, Lee K, Toutanova K (2018) Bert: pre-training of deep bidirectional transformers for language understanding. Preprint arXiv:1810.04805
Titov I, McDonald R (2008) Modeling online reviews with multi-grain topic models. In: Proceedings of the 17th international conference on World Wide Web, pp 111–120
Mukherjee A, Liu B (2012) Aspect extraction through semi-supervised modeling. In: Proceedings of the 50th annual meeting of the association for computational linguistics: Long papers-volume 1. Association for Computational Linguistics, pp 339–348
Yan X, Guo J, Lan Y, Cheng X (2013) A biterm topic model for short texts. In: Proceedings of the 22nd international conference on World Wide Web, pp 1445–1456
Zbontar J, Jing L, Misra I, LeCun Y, Deny S (2021) Barlow twins: self-supervised learning via redundancy reduction
Lara JS, González FA (2020) Dissimilarity mixture autoencoder for deep clustering. Preprint arXiv:2006.08177
Hadifar A, Sterckx L, Demeester T, Develder C (2019) A self-training approach for short text clustering. In: Proceedings of the 4th workshop on representation learning for NLP (RepL4NLP-2019), pp 194–199
Zhang W, Dong C, Yin J, Wang J (2019)Attentive representation learning with adversarial training for short text clustering. Preprint arXiv:1912.03720
García-Pablos A, Cuadros M, Rigau G (2018) W2vlda: almost unsupervised system for aspect based sentiment analysis. Exp Syst Appl 91:127–137
Angelidis S, Lapata M (2018) Summarizing opinions: aspect extraction meets sentiment prediction and they are both weakly supervised. Preprint arXiv:1808.08858
Sokhin T, Khodorchenko M, Butakov N (2020) Unsupervised neural aspect search with related terms extraction. Preprint arXiv:2005.02771
Szegedy C, Zaremba W, Sutskever I, Bruna J, Erhan D, Goodfellow I, Fergus R (2013) Intriguing properties of neural networks. Preprint arXiv:1312.6199
Goodfellow IJ, Shlens J, Szegedy C (2014) Explaining and harnessing adversarial examples. Preprint arXiv:1412.6572
Wu Y, Schuster M, Chen Z, Le QV, Norouzi M, Macherey W, Krikun M, Cao Y, Gao Q, Macherey K, et al. (2016) Google’s neural machine translation system: bridging the gap between human and machine translation. Preprint arXiv:1609.08144
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, pp 5998–6008
Xiao H (2018) bert-as-service. https://github.com/hanxiao/bert-as-service
Socher R, Karpathy A, Le QV, Manning CD, Ng AY (2014) Grounded compositional semantics for finding and describing images with sentences. Trans Assoc Comput Linguist 2:207–218
Weston J, Bengio S, Usunier N (2011) Wsabie: scaling up to large vocabulary image annotation. In: Proceedings of the twenty-second international joint conference on artificial intelligence—volume volume three, IJCAI’11. AAAI Press, pp 2764–2770
Iyyer M, Guha A, Chaturvedi S, Boyd-Graber J, Daumé III H (2016) Feuding families and former Friends: unsupervised learning for dynamic fictional relationships. In: Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies, San Diego, California, June . Association for Computational Linguistics, pp 1534–1544
Miyato T, Maeda S-i, Koyama M, Nakae K, Ishii S (2015) Distributional smoothing with virtual adversarial training. Preprint arXiv:1507.00677
Wang L, Liu K, Cao Z, Zhao J, De Melo G (2015)Sentiment-aspect extraction based on restricted boltzmann machines. In: Proceedings of the 53rd annual meeting of the association for computational linguistics and the 7th international joint conference on natural language processing (volume 1: long papers), pp 616–625
He R, McAuley J (2016) Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In: proceedings of the 25th international conference on world wide web, pp 507–517
Li N, Chow C-Y, Zhang J-D (2019) Seeded-btm: enabling biterm topic model with seeds for product aspect mining. In: 2019 IEEE 21st international conference on high performance computing and communications; IEEE 17th international conference on smart city; IEEE 5th international conference on data science and systems (HPCC/SmartCity/DSS). IEEE, pp 2751–2758
Goldberg Y, Levy O (2014) word2vec explained: deriving mikolov et al.’s negative-sampling word-embedding method. Preprint arXiv:1402.3722
Vargas DS, Pessutto LRC, Moreira VP (2020)Simple unsupervised similarity-based aspect extraction. Preprint arXiv:2008.10820
Luo L, Ao X, Song Y, Li J, Yang X, He Q, Yu D (2019)Unsupervised neural aspect extraction with sememes. In: IJCAI, pp 5123–5129
Xu H, Liu B, Shu L, Yu PS (2019) Bert post-training for review reading comprehension and aspect-based sentiment analysis. Preprint arXiv:1904.02232
Kingma DP, Ba J (2014) Adam: a method for stochastic optimization. Preprint arXiv:1412.6980
Conneau A, Khandelwal K, Goyal N, Chaudhary V, Wenzek G, Guzmán F, Grave E, Ott M, Zettlemoyer L, Stoyanov V (2019) Unsupervised cross-lingual representation learning at scale. Preprint arXiv:1911.02116
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors have no relevant financial or non-financial interests to disclose. The authors have no conflicts of interest to declare that are relevant to the content of this article. All authors certify that they have no affiliations with or involvement in any organization or entity with any financial interest or non-financial interest in the subject matter or materials discussed in this manuscript. The authors have no financial or proprietary interests in any material discussed in this article.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Kumar, A., Gupta, P., Kotak, N. et al. BARLAT: A Nearly Unsupervised Approach for Aspect Category Detection. Neural Process Lett 54, 4495–4519 (2022). https://doi.org/10.1007/s11063-022-10819-4
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11063-022-10819-4