AbCoRD: Exploiting multimodal generative approach for Aspect-based Complaint and Rationale Detection
Pages 8571 - 8579
Abstract
Valuable feedback can be found in customer reviews about a product or service, but it can be difficult to identify specific complaints and their underlying causes from the text. While various methods have been used to detect and analyze complaints, there has been little research on examining complaints at the aspect level and the reasons behind these complaints. To address this, we added rationale annotation for aspect-based complaint classes to a publicly available benchmark multimodal complaint dataset (CESAMARD) covering five domains (books, electronics, edibles, fashion, and miscellaneous). Current methods treat these tasks as classification problems and do not use external knowledge. Our study proposes a knowledge-infused, hierarchical, multimodal generative approach for aspect-based complaint and rationale detection that reframes the multitasking problem as a multimodal text-to-text generation task. Our approach achieved benchmark performance in the aspect-based complaint and rationale detection task through an extensive evaluation. We demonstrated that our model consistently outperformed all other baselines and state-of-the-art models in both full and few-shot settings. Our study contributes to the development of more accurate and efficient methods for extracting valuable insights from customer reviews to improve products and service 1
References
[1]
Md Shad Akhtar, Asif Ekbal, and Pushpak Bhattacharyya. 2016. Aspect based sentiment analysis: category detection and sentiment classification for Hindi. In International Conference on Intelligent Text Processing and Computational Linguistics. Springer, 246--257.
[2]
Kazuki Akiyama, Akihiro Tamura, and Takashi Ninomiya. 2021. Hie-BART: Document Summarization with Hierarchical BART. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Student Research Workshop. Association for Computational Linguistics, Online, 159--165. https://doi.org/10.18653/v1/2021.naacl-srw.20
[3]
Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Computational linguistics, Vol. 34, 4 (2008), 555--596.
[4]
Shreesh Bhat and Aron Culotta. 2017. Identifying leading indicators of product recalls from online reviews using positive unlabeled learning and domain adaptation. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 11. 480--483.
[5]
Penelope Brown, Stephen C Levinson, and Stephen C Levinson. 1987. Politeness: Some universals in language usage. Vol. 4. Cambridge university press.
[6]
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, Vol. 33 (2020), 1877--1901.
[7]
Kristof Coussement and Dirk Van den Poel. 2008. Improving customer complaint management by automatic email classification using linguistic style features as predictors. Decision Support Systems, Vol. 44, 4 (2008), 870--882.
[8]
Michael Crawshaw. 2020. Multi-task learning with deep neural networks: A survey. arXiv preprint arXiv:2009.09796 (2020).
[9]
Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin, Vol. 76, 5 (1971), 378.
[10]
OO Iyiola and OS Ibidunni. 2013. The relationship between complaints, emotion, anger, and subsequent behavior of customers. IOSR Journal of Humanities and Social Sciences, Vol. 17, 6 (2013), 34--41.
[11]
William M Jenkins and Joseph P Cangemi. 1979. Levels of Intensity of Dissatisfaction: A Model. Education, Vol. 99, 4 (1979).
[12]
Mali Jin and Nikolaos Aletras. 2021. Modeling the Severity of Complaints in Social Media. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tür, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (Eds.). Association for Computational Linguistics, 2264--2274. https://doi.org/10.18653/v1/2021.naacl-main.180
[13]
Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2020. SpanBERT: Improving Pre-training by Representing and Predicting Spans. arxiv: 1907.10529 [cs.CL]
[14]
Douwe Kiela, Suvrat Bhooshan, Hamed Firooz, Ethan Perez, and Davide Testuggine. 2020. Supervised Multimodal Bitransformers for Classifying Images and Text. arxiv: 1909.02950 [cs.CL]
[15]
M Lailiyah, S Sumpeno, and IK E Purnama. 2017. Sentiment analysis of public complaints using lexical resources between Indonesian sentiment lexicon and Sentiwordnet. In 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA). IEEE, 307--312.
[16]
Darren Law, Richard Gruss, and Alan S. Abrahams. 2017. Automated defect discovery for dishwasher appliances from online consumer reviews. Expert Syst. Appl., Vol. 67 (2017), 84--94. https://doi.org/10.1016/j.eswa.2016.08.069
[17]
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. arXiv preprint arXiv:1910.13461 (2019).
[18]
Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. 2019. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems, Vol. 32 (2019).
[19]
E Olshtain and L Weinbach. 1985. Complaints: A Study of Speech Act Behavior among Native and Nonnative Speakers of Hebrew. The Prag-matic Perspective.
[20]
Soujanya Poria, Devamanyu Hazarika, Navonil Majumder, Gautam Naik, Erik Cambria, and Rada Mihalcea. 2018. Meld: A multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508 (2018).
[21]
Soujanya Poria, Navonil Majumder, Devamanyu Hazarika, Deepanway Ghosal, Rishabh Bhardwaj, Samson Yu Bai Jian, Pengfei Hong, Romila Ghosh, Abhinaba Roy, Niyati Chhaya, et al. 2021. Recognizing emotion cause in conversations. Cognitive Computation, Vol. 13 (2021), 1317--1332.
[22]
Daniel Preotiuc-Pietro, Mihaela Gaman, and Nikolaos Aletras. 2019. Automatically Identifying Complaints in Social Media. In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, Anna Korhonen, David R. Traum, and Lluís Màrquez (Eds.). Association for Computational Linguistics, 5008--5019. https://doi.org/10.18653/v1/p19-1495
[23]
Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, Vol. 1, 8 (2019), 9.
[24]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. https://doi.org/10.48550/ARXIV.1910.10683
[25]
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. J. Mach. Learn. Res., Vol. 21 (2020), 140:1--140:67. http://jmlr.org/papers/v21/20-074.html
[26]
Pranav Rajpurkar, Jian Zhang, Konstantin Lopyrev, and Percy Liang. 2016. Squad: 100,000 questions for machine comprehension of text. arXiv preprint arXiv:1606.05250 (2016).
[27]
Tulika Saha, Apoorva Upadhyaya, Sriparna Saha, and Pushpak Bhattacharyya. 2021. Towards Sentiment and Emotion aided Multi-modal Speech Act Classification in Twitter. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 5727--5737.
[28]
Apoorva Singh, Soumyodeep Dey, Anamitra Singha, and Sriparna Saha. 2022a. Sentiment and Emotion-Aware Multi-Modal Complaint Identification. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022. AAAI Press, 12163--12171. https://ojs.aaai.org/index.php/AAAI/article/view/21476
[29]
Apoorva Singh, Vivek Gangwar, Shubham Sharma, and Sriparna Saha. 2023. Knowing What and How: A Multi-modal Aspect-Based Framework for Complaint Detection. In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 2023, Dublin, Ireland, April 2--6, 2023, Proceedings, Part II. Springer, 125--140.
[30]
Apoorva Singh, Arousha Nazir, and Sriparna Saha. 2022b. Adversarial Multi-task Model for Emotion, Sentiment, and Sarcasm Aided Complaint Detection. In Advances in Information Retrieval - 44th European Conference on IR Research, ECIR 2022, Stavanger, Norway, April 10-14, 2022, Proceedings, Part I (Lecture Notes in Computer Science, Vol. 13185), Matthias Hagen, Suzan Verberne, Craig Macdonald, Christin Seifert, Krisztian Balog, Kjetil Nørvåg, and Vinay Setty (Eds.). Springer, 428--442. https://doi.org/10.1007/978-3-030-99736-6_29
[31]
Apoorva Singh and Sriparna Saha. 2021. Are You Really Complaining? A Multi-task Framework for Complaint Identification, Emotion, and Sentiment Classification. In International Conference on Document Analysis and Recognition. Springer, 715--731.
[32]
Rohit Sridhar and Diyi Yang. 2022. Explaining Toxic Text via Knowledge Enhanced Text Generation. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, Seattle, United States, 811--826. https://doi.org/10.18653/v1/2022.naacl-main.59
[33]
Anna Trosborg. 2011. Interlanguage pragmatics: Requests, complaints, and apologies. Vol. 7. Walter de Gruyter.
[34]
Camilla Vásquez. 2011. Complaints online: The case of TripAdvisor. Journal of Pragmatics, Vol. 43, 6 (2011), 1707--1717.
[35]
Bernard L Welch. 1947. The generalization of ?STUDENT'S'problem when several different population varlances are involved. Biometrika, Vol. 34, 1--2 (1947), 28--35.
[36]
Sen Wu. 2020. Automating Knowledge Distillation and Representation from Richly Formatted Data. Stanford University.
[37]
Wei Yang, Luchen Tan, Chunwei Lu, Anqi Cui, Han Li, Xi Chen, Kun Xiong, Muzi Wang, Ming Li, Jian Pei, and Jimmy Lin. 2019. Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 2 (Industry Papers), Anastassia Loukina, Michelle Morales, and Rohit Kumar (Eds.). Association for Computational Linguistics, 56--63. https://doi.org/10.18653/v1/n19-2008
[38]
Ron Zhu. 2020. Enhance Multimodal Transformer With External Label And In-Domain Pretrain: Hateful Meme Challenge Winning Solution.
Index Terms
- AbCoRD: Exploiting multimodal generative approach for Aspect-based Complaint and Rationale Detection
Recommendations
Investigating the Impact of Multimodality and External Knowledge in Aspect-level Complaint and Sentiment Analysis
CIKM '23: Proceedings of the 32nd ACM International Conference on Information and Knowledge ManagementAutomated complaint analysis is vital for generating critical insights, which in turn enhance customer satisfaction, product quality, and overall business performance. Nevertheless, conventional methods frequently fail to capture the nuances of aspect-...
Aspect-Based Complaint and Cause Detection: A Multimodal Generative Framework with External Knowledge Infusion
Machine Learning and Knowledge Discovery in Databases: Applied Data Science and Demo TrackAbstractCustomer reviews often contain valuable feedback about a product or service, but it can be challenging to extract specific complaints and their underlying causes from the text. Despite the use of various methods to detect and analyze complaints, ...
Seeing Beyond Words: Multimodal Aspect-Level Complaint Detection in Ecommerce Videos
MM '24: Proceedings of the 32nd ACM International Conference on MultimediaComplaints are pivotal expressions within e-commerce communication, yet the intricate nuances of human interaction present formidable challenges for AI agents to grasp comprehensively. While recent attention has been drawn to analyzing complaints within ...
Comments
Information & Contributors
Information
Published In
October 2023
9913 pages
ISBN:9798400701085
DOI:10.1145/3581783
- General Chairs:
- Abdulmotaleb El Saddik,
- Tao Mei,
- Rita Cucchiara,
- Program Chairs:
- Marco Bertini,
- Diana Patricia Tobon Vallejo,
- Pradeep K. Atrey,
- M. Shamim Hossain
Copyright © 2023 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 27 October 2023
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
MM '23
Sponsor:
MM '23: The 31st ACM International Conference on Multimedia
October 29 - November 3, 2023
Ottawa ON, Canada
Acceptance Rates
Overall Acceptance Rate 2,145 of 8,556 submissions, 25%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 111Total Downloads
- Downloads (Last 12 months)45
- Downloads (Last 6 weeks)3
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in