Skip to main content
SpringerLink
Log in

Cross-domain aspect-based sentiment analysis using domain adversarial training

  • Published:
World Wide Web Aims and scope Submit manuscript

Abstract

Over the last decades, the increasing popularity of the Web came together with an extremely large volume of reviews on products and services useful for both companies and customers to adjust their behaviour with respect to the expressed opinions. Given this growth, Aspect-Based Sentiment Analysis (ABSA) has turned out to be an important tool required to understand people’s preferences. However, despite the large volume of data, the lack of data annotations restricts the supervised ABSA analysis to only a limited number of domains. To tackle this problem a transfer learning strategy is implemented by extending the state-of-the-art LCR-Rot-hop++ model for ABSA with the methodology of Domain Adversarial Training (DAT). The output is a cross-domain deep learning structure, called DAT-LCR-Rot-hop++. The major advantage of DAT-LCR-Rot-hop++ is the fact that it does not require any labeled target domain data. The results are obtained for six different domain combinations with testing accuracies ranging from 35% up until 74%, showing both the limitations and benefits of this approach. Once DAT-LCR-Rot-hop++ is able to find the similarities between domains, it produces good results. However, if the domains are too distant, it is not capable of generating domain-invariant features. This result is amplified by our additional analysis to add the neutral aspects to the positive or negative class. The performance of DAT-LCR-Rot-hop++ is very dependent on the similarity between distributions of source and target domain and the presence of a dominant sentiment class in the training set.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Algorithm 1
Fig. 2

Similar content being viewed by others

Availability of data and materials

The code and the dataset are available at https://github.com/jorisknoester/DAT-LCR-Rot-hop-PLUS-PLUS.

References

  1. Bergstra, J., Bardenet, R., Bengio, Y., Kégl, B.: Algorithms for hyper-parameter optimization. In: 25th Annual conference on neural information processing systems (NIPS 2011), pp. 2546–2554. Curran Associates (2011)

  2. Brauwers, G., Frasincar, F.: A survey on aspect-based sentiment classification. ACM Comput. Surv. 55(4), 65:1-65:37 (2023)

    Article  Google Scholar 

  3. Chapman, B.: Gamestop: reddit users claim victory as $13bn hedge fund closes position, accepting huge losses (2021). https://www.independent.co.uk/news/business/gamestop-share-price-reddit-hedge-fund-melvin-capital-b1793543.html

  4. Chen, Y., Tong, Z., Zheng, Y., Samuelson, H., Norford, L.: Transfer learning with deep neural networks for model predictive control of HVAC and natural ventilation in smart buildings. J. Clean. Prod. 254, 119866 (2020)

    Article  Google Scholar 

  5. Ciresan, D., Meier, U., Schmidhuber, J.: Multi-column deep neural networks for image classification. In: 2012 IEEE Conference on computer vision and pattern recognition (CVPR 2012), vol. 1, pp. 3642–3649. IEEE Computer Society (2012)

  6. Devlin, J., Chang, K., Lee, K., Huang, D., Toutanova, K.: Bert: Pre-training of deep bidirectional transformers for language understanding. In: 2019 Conference of the North American chapter of the association for computational linguistics (NAACL-HLT 2019), pp. 4171–4186. ACL (2019)

  7. Ganin, Y., Lempitsky, V.: Unsupervised domain adaption by backpropagation. In: 32nd International conference on machine learning (ICML 2015), vol. 37, pp. 1180–1189. PMLR (2015)

  8. Ganin, Y., Ustinova, E., Ajakan, H., Germain, P., Larochelle, H., Laviolette, F., Marchand, M., Lempitsky, V.: Domain-adversarial training of neural networks. J. Mach. Learn. Res. 17, 59:10-59:35 (2016)

    MathSciNet  Google Scholar 

  9. Goodfellow, I., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial networks. In: 28th Annual conference on neural information processing systems (NIPS 2014), pp. 2672–2680 (2014)

  10. Hendricks, D.: Complete history of social media: then and now (2013). https://smallbiztrends.com/2013/05/the-complete-history-of-social-media-infographic.html

  11. Hong, W., Wang, Z., Yang, M., Yuan, J.: Conditional generative adversarial network for structured domain adaption. In: 8th IEEE International conference on computer vision and pattern recognition (CVPR 2018), vol. 12, pp. 1335–1344. IEEE (2018)

  12. Hong, Y., Zhou, W., Zhang, J., Zhu, Q., Zhou, G.: Self-regulation: employing a generative adversarial network to improve event detection. In: 56th Annual meeting of the association for computational linguistics (ACL 2018), pp. 515–526. ACL (2018)

  13. Johansson, F., Shalit, U., Sontag, D.: Learning representations for counterfactual inference. In: 33rd International Conference on Machine Learning (ICML 2016). JMLR Workshop and conference proceedings, vol. 48, pp. 3020–3029. JMLR (2016)

  14. Kamath, S., Gupta, S., Carvalho, V.: Reversing gradients in adversarial domain adaption for question deduplication and textual entailment tasks. In: 57th Annual meeting of the association for cumputational linguistics (ACL 2019), pp. 5545–5550. ACL (2019)

  15. Knoester, J., Frasincar, F., Trusca, M.M.: Domain adversarial training for aspect-based sentiment analysis. In: 23rd International conference on Web information systems engineering (WISE 2022). LNCS, vol. 13724, pp. 21–37. Springer (2022)

  16. Lai, S., Xu, L., Liu, K., Zhao, J.: Recurrent convolutional neural networks for text classification. In: 29th Conference on artificial intelligence (AAAI 2015), vol. 29, pp. 2267–2273. AAAI Press (2015)

  17. Liu, C., Belkin, M.: Accelerating SGD with momentum for over-parameterized learning. In: 8th International conference on learning representations (ICLR, 2020). OpenReview.net (2020)

  18. Álvarez López, T., Fernández-Gavilanes, M., Costa-Montenegro, E., Bellot, P.: A proposal for book oriented aspect based sentiment analysis: comparison over domains. In: 23rd International conference on applications of natural language to information systems (NLDB 2018). LNCS, vol. 10859, pp. 3–14. Springer (2018)

  19. Maat, E.D., Krabben, K., Winkels, R.: Machine learning versus knowledge based classification of legal texts. In: 23rd Annual conference on legal knowledge and information systems (JURIX 2010), vol. 223, pp. 87–96. IOS Press (2010)

  20. Mauro, M., Mazzia, V., Khalil, A., Chiaberge, M.: Domain-adversarial training of self-attention based networks for land cover classification using multi-temporal sentinel-2 satellite imagery. Remote Sens. 13(13), 2564 (2021)

    Article  Google Scholar 

  21. Pan, S., Yang, Q.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2009)

    Article  Google Scholar 

  22. Pang, B., Lee, L.: A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In: 42nd Annual meeting of the association for computational linguistics (ACL 2004), pp. 271–278. ACL (2004)

  23. Pang, B., Lee, L., Vaithyanathan, S.: Thumbs up? sentiment classification using machine learning techniques. In: 2002 Conference on empirical methods in natural language processing 2002 (EMNLP 2002), pp. 79–86. ACL (2002)

  24. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: Semeval-2015 task 12: aspect based sentiment analysis. In: 9th International workshop on semantic evaluation (SemEval 2015), pp. 486–495. ACL (2015)

  25. Pontiki, M., Galanis, D., Papageorgiou, H., Androutsopoulos, I., Manandhar, S., Al-Smadi, M., Al-Ayyoub, M., Zhao, Y., Qin, B., De Clercq, O., Hoste, V., Apidianaki, M., Tannier, X., Loukachevitch, N., Kotelnikov, E., Bel, N., Zafra, S., Eryigit, G.: Semeval-2016 task 5: aspect based sentiment analysis. In: 10th International workshop on semantic evaluation (SemEval 2016), pp. 19–30. ACL (2016)

  26. Pontiki, M., Galanis, D., Pavlopoulos, J., Papageorgiou, H., Androutsopoulos, I., Manandhar, S.: Semeval-2014 task 4: aspect based sentiment analysis. In: 8th International workshop on semantic evaluation (SemEval 2014), pp. 27–35. ACL (2014)

  27. Schouten, K., Frasincar, F.: Survey on aspect-level sentiment analysis. IEEE Trans. Knowl. Data Eng. 28, 813–880 (2016)

    Article  Google Scholar 

  28. Schouten, K., Frasincar, F.: Ontology-driven sentiment analysis of product and service aspects. In: 15th International conference of European semantic Web (ESWC 2018). LNCS, vol. 10843, pp. 608–623. Springer (2018)

  29. Taboada, M., Brooke, J., Tofiloski, M., Voll, K., Stede, M.: Lexicon-based methods for sentiment analysis. Comput. Linguist. 37, 267–307 (2011)

    Article  Google Scholar 

  30. Tankovska, H.: Social media - statistics and facts (2021). https://www.statista.com/topics/1164/social-networks/

  31. Thet, T., Na, J., Khoo, C.: Aspect-based sentiment analysis of movie reviews on discussion boards. J. Inf. Sci. 36, 823–848 (2010)

    Article  Google Scholar 

  32. Towell, G., Shavlik, J.: Knowledge-based artificial neural networks. Artifical. Intelligence 70, 119–165 (1994)

    Google Scholar 

  33. Trusca, M., Wassenberg, D., Frasincar, F., Dekker, R.: A hybrid approach for aspect-based sentiment analysis using deep contextual word embeddings and hierarchical attention. In: 20th International conference on Web engineering (ICWE 2020). LNCS, vol. 12128, pp. 365–380. Springer (2020)

  34. Wallaart, O., Frasincar, F.: A hybrid approach for aspect-based sentiment analysis using a lexicalized domain ontology and attentional neural models. In: 16th International conference of European semantic Web (ESWC 2019). LNCS, vol. 11503, pp. 363–378. Springer (2019)

  35. Wang, F., Zhang, Q.: Knowledge-based neural models for microwave design. IEEE Transactions on Microwaves Theory and Techniques 45, 2333–2343 (1997)

    Article  Google Scholar 

  36. Wang, Z., Huang, M., Zhao, L., Zhu, X.: Attention-based LSTM for aspect-level sentiment classification. In: 2016 Conference on empirical methods in natural language processing (EMNLP 2016), pp. 606–615. ACL (2016)

  37. Wu, Y., Inkpen, D., El-Roby, A.: Co-Regularized Adversarial Learning for Multi-Domain Text Classification. In: 2022 International conference on artificial intelligence and statistics (AISTATS 2022), pp. 6690–6701. PMLR (2022)

  38. Yanase, T., Yanai, K., Sato, M., Miyoshi, T., Niwa, Y.: bunji at SemEval-2016 task 5: neural and synctactic models of entity-attribute relationship for aspect-based sentiment analysis. In: 10th International workshop on semantic evaluation (SemEval 2016), pp. 289–295. ACL (2016)

  39. Yosinski, J., Clune, J., Bengio, Y., Lipson, H.: How transferable are features in deep neural networks? In: 27th Annual conference on neural information processing systems (NIPS 2014), vol. 27, pp. 3320–3328. Curran Associates (2014)

  40. Yuan, J., Zhao, Y., Qin, B., Liu, T.: Learning to share by masking the non-shared for multi-domain sentiment classification. Int. J. Mach. Learn. Cybern. 13(9), 2711–2724 (2021)

    Article  Google Scholar 

  41. Zhang, W., Ouyang, W., Li, W., Xu, D.: Collaborative and adversarial network for unsupervised domain adaption. In: 2018 Conference on computer vision and pattern recognition (CVPR 2018), pp. 3801–3809. IEEE (2018)

  42. Zhang, Y., Qiu, Z., Yao, T., Liu, D., Mei, T.: Fully convolutional adaption networks for semantic segmentation. In: 2018 International conference on computer vision and pattern recognition (CVPR 2018), pp. 6810–6818. IEEE (2018)

  43. Zheng, L., Zhang, Y., Wu, Y., Wei, Y., Yang, Q.: End-to-end adversarial memory network for cross-domain sentiment classification. In: 26th International joint conference on artificial intelligence (IJCAI 2017), pp. 2237–2243. IJCAI (2017)

  44. Zheng, S., Xia, R.: Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention. arXiv preprint arXiv:1802.00892 (2018)

Download references

Funding

Not applicable.

Author information

Authors and Affiliations

  1. Department of Econometrics, Erasmus University Rotterdam, Burgemeester Oudlaan 50, Rotterdam, 3062 PA, the Netherlands

    Joris Knoester & Flavius Frasincar

  2. Department and Informatics and Economic Cybernetics, Bucharest University of Economic Studies, Piata Romana, Bucharest, 010374, Romania

    Maria Mihaela Truşcǎ

Authors
  1. Joris Knoester
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Flavius Frasincar
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Maria Mihaela Truşcǎ
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

Joris Knoester did the implementation. Flavius Flasincar and Maria Mihaela Trusca did review the manuscript. All authors wrote the manuscript.

Corresponding author

Correspondence to Maria Mihaela Truşcǎ.

Ethics declarations

Ethical Approval

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This article belongs to the Topical Collection: Special Issue on Web Information Systems Engineering 2022

Guest Editors: Richard Chbeir, Helen Huang, Yannis Manolopoulos and Fabrizio Silvestri.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Knoester, J., Frasincar, F. & Truşcǎ, M.M. Cross-domain aspect-based sentiment analysis using domain adversarial training. World Wide Web 26, 4047–4067 (2023). https://doi.org/10.1007/s11280-023-01217-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11280-023-01217-4

Keywords

Search

Navigation