Skip to main content
SpringerLink
Log in

Multitask Learning for Complaint Identification and Sentiment Analysis

  • Published:
Cognitive Computation Aims and scope Submit manuscript

Abstract

In today’s competitive business world, customer service is often at the heart of businesses that can help strengthen their brands. Resolution of customers’ complaints in a timely and efficient manner is key to improving customer satisfaction. Moreover, customers’ complaints play an important role in identifying their requirements which offer a starting point for effective and efficient planning of companies’ overall R&D and new product or service development activities. Having said that, organizations encounter challenges towards automatically identifying complaints buried deep in massive online content. Our current work centers around learning two closely related tasks, viz. complaint identification and sentiment classification. We leverage weak supervision to annotate the corpus with sentiment labels. We propose a deep multitask framework that features a knowledge element that uses AffectiveSpace to infuse commonsense knowledge specific features into the learning process. The framework models complaint identification (the primary task) and sentiment classification (supplementary task) simultaneously. Experimental results show that our proposed multitask system obtains the highest cross-validation accuracy of 83.73 +/- 1.52 % for the complaint identification task and 69.01 +/- 1.74 % for the sentiment classification task. Our proposed multitask system outperforms the single-task systems indicating a strong correlation between sentiment analysis and complaint classification tasks, thus benefiting from each other when learned concurrently.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

Notes

  1. https://github.com/danielpreotiuc/complaints-social-media

  2. food&beverage, apparel, retail, cars, services, software, transport, electronics, and other

  3. https://github.com/cjhutto/vaderSentiment

  4. https://textblob.readthedocs.io/en/dev/

  5. https://www.nltk.org/api/nltk.sentiment.html

  6. https://sentic.net/downloads/

  7. GloVe:http://nlp.stanford.edu/data/wordvecs/glove.840B.300d.zip

  8. a high-level neural networks API: https://keras.io/

  9. https://sentic.net/downloads/

  10. https://keras.io/

  11. https://scikit-learn.org/stable/

  12. using loss_weights parameter of Keras compile function

  13. We perform Student’s t-test for assessing the statistical significance

References

  1. Olshtain E, Weinbach L. 10. complaints: A study of speech act behavior among native and non-native speakers of hebrew. In The pragmatic perspective. John Benjamins, 1987. p. 195.

  2. Wang L, Niu J, Song H, Atiquzzaman M. Sentirelated: A cross-domain sentiment classification algorithm for short texts through sentiment related index. Journal of Network and Computer Applications. 2018;101:111–9.

    Article  Google Scholar 

  3. Liu B, Zhang L. A survey of opinion mining and sentiment analysis. In C. C. Aggarwal and C. Zhai, editors, Mining Text Data, p. 415–463. Springer, 2012. https://doi.org/10.1007/978-1-4614-3223-4_13

  4. Vásquez C. Complaints online: The case of tripadvisor. Journal of Pragmatics. 2011;43(6):1707–17.

    Article  Google Scholar 

  5. Caruana R, De Sa VR. Promoting poor features to supervisors: Some inputs work better as outputs. In Adv Neural Inf Proces Syst. 1997. p 389–395.

  6. Ruder S. An overview of multi-task learning in deep neural networks. CoRR, abs/1706.05098, 2017.

  7. Cambria E, Li Y, Xing FZ, Poria S, Kwok K. Senticnet 6: Ensemble application of symbolic and subsymbolic AI for sentiment analysis. In M. d’Aquin, S. Dietze, C. Hauff, E. Curry, and P. Cudré-Mauroux, editors, CIKM ’20: The 29th ACM International Conference on Information and Knowledge Management, Virtual Event, Ireland, October 19-23, p. 105–114. ACM, 2020. https://doi.org/10.1145/3340531.3412003

  8. Preotiuc-Pietro D, Gaman M, Aletras N. Automatically identifying complaints in social media. In A. Korhonen, D. R. Traum, and L. Màrquez, editors, Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers. Association for Computational Linguistics, 2019. p. 5008–5019.

  9. Liu W, Wang Z, Liu X, Zeng N, Liu Y, Alsaadi FE. A survey of deep neural network architectures and their applications. Neurocomputing. 2017;234:11–26.

    Article  Google Scholar 

  10. Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep convolutional neural networks. In Adv Neural Inf Proces Syst. 2012. p. 1097–1105.

  11. Dashtipour K, Poria S, Hussain A, Cambria E, Hawalah AY, Gelbukh A, Zhou Q. Multilingual sentiment analysis: state of the art and independent comparison of techniques. Cogn Comput. 2016;8(4):757–71.

    Article  Google Scholar 

  12. Xia Y, Cambria E, Hussain A, Zhao H. Word polarity disambiguation using bayesian model and opinion-level features. Cogn Comput. 2015;7(3):369–80.

    Article  Google Scholar 

  13. Zhang X, Zhao J, LeCun Y. Character-level convolutional networks for text classification. In Advances in neural information processing systems. 2015. p. 649–657.

  14. Xu J, Chen D, Qiu X, Huang X. Cached long short-term memory neural networks for document-level sentiment classification. CoRR, abs/1610.04989, 2016.

  15. Jiang L, Yu M, Zhou M, Liu X, Zhao T. Target-dependent twitter sentiment classification. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies. 2011. p 151–160.

  16. Yang M, Tu W, Wang J, Xu F, Chen X. Attention-based lstm for target-dependent sentiment classification. In Proceedings of the thirty-first AAAI conference on artificial intelligence. 2017. p. 5013–5014.

  17. Wang S, Wu B, Wang B, Tong X. Complaint classification using hybrid-attention GRU neural network. In Q. Yang, Z. Zhou, Z. Gong, M. Zhang, and S. Huang, editors, Advances in Knowledge Discovery and Data Mining - 23rd Pacific-Asia Conference, PAKDD 2019, Macau, China, April 14-17, 2019, Proceedings, Part I, volume 11439 of Lecture Notes in Computer Science. Springer, 2019. p. 251–262. https://doi.org/10.1007/978-3-030-16148-4\_20

  18. Yang Z, Yang D, Dyer C, He X, Smola A, Hovy E. Hierarchical attention networks for document classification. In Proceedings of the 2016 conference of the North American chapter of the association for computational linguistics: human language technologies. 2016. p. 1480–1489.

  19. Zhou C, Sun C, Liu Z, Lau FCM. A C-LSTM neural network for text classification. CoRR, abs/1511.08630, 2015.

  20. Mathur P, Sawhney R, Ayyar M, Shah R. Did you offend me? classification of offensive tweets in hinglish language. In Proceedings of the 2nd Workshop on Abusive Language Online (ALW2). 2018. p. 138–148.

  21. Assawinjaipetch P, Shirai K, Sornlertlamvanich V, Marukata S. Recurrent neural network with word embedding for complaint classification. In Proceedings of the Third International Workshop on Worldwide Language Service Infrastructure and Second Workshop on Open Infrastructures and Analysis Frameworks for Human Language Technologies (WLSI/OIAF4HLT2016). 2016. p. 36–43.

  22. Akhtar MS, Ekbal A, Cambria E. How intense are you? predicting intensities of emotions and sentiments using stacked ensemble. IEEE Comput Intell Mag. 2020;15(1):64–75.

    Article  Google Scholar 

  23. Cambria E. Affective computing and sentiment analysis. IEEE Intell Syst. 2016;31(2):102–7.

    Article  Google Scholar 

  24. Cohen AD, Olshtain E. The production of speech acts by efl learners. Tesol Quarterly. 1993;27(1):33–56.

    Article  Google Scholar 

  25. Boxer D. Social distance and speech behavior: The case of indirect complaints. J Pragmat. 1993;19(2):103–25.

    Article  Google Scholar 

  26. Brown P, Levinson SC, Levinson SC. Politeness: Some universals in language usage, vol. 4. Cambridge University Press; 1987.

  27. Trosborg A. Interlanguage pragmatics: Requests, complaints, and apologies, vol. 7. Walter de Gruyter; 2011.

  28. Laforest M. Scenes of family life: Complaining in everyday conversation. J Pragmat. 2002;34(10–11):1595–620.

    Article  Google Scholar 

  29. Hartford B, Mahboob A. Models of discourse in the letter of complaint. World Englishes. 2004;23(4):585–600.

    Article  Google Scholar 

  30. Ranosa-Madrunio M. The discourse organization of letters of complaint to editors in philippine english and singapore english. Philippine Journal of Linguistics. 2004;35(2):67–97.

    Google Scholar 

  31. Meinl ME. Electronic complaints: an empirical study on British English and German complaints on eBay. PhD thesis, University of Bonn, 2010.

  32. Pryzant R, Shen K, Jurafsky D, Wagner S. Deconfounded lexicon induction for interpretable social science. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers). 2018. p. 1615–1625.

  33. Yang W, Tan L, Lu C, Cui A, Li H, Chen X, Xiong K, Wang M, Li M, Pei J, et al. Detecting customer complaint escalation with recurrent neural networks and manually-engineered features. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Industry Papers). 2019. p. 56–63.

  34. Deep KS, Akhtar MS, Ekbal A, Bhattacharyya P. Related tasks can share! A multi-task framework for affective language. CoRR, abs/2002.02154, 2020.

  35. Cambria E, Hussain A. Sentic album: content-, concept-, and context-based online personal photo management system. Cogn Comput. 2012;4(4):477–96.

    Article  Google Scholar 

  36. Grassi M, Cambria E, Hussain A, Piazza F. Sentic web: A new paradigm for managing social media affective information. Cogn Comput. 2011;3(3):480–9.

    Article  Google Scholar 

  37. Ma Y, Peng H, Cambria E. Targeted aspect-based sentiment analysis via embedding commonsense knowledge into an attentive LSTM. In S. A. McIlraith and K. Q. Weinberger, editors, Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence (EAAI-18), New Orleans, Louisiana, USA, February 2-7, 2018, p. 5876–5883. AAAI Press, 2018. https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16541

  38. Li Y, Pan Q, Yang T, Wang S, Tang J, Cambria E. Learning word representations for sentiment analysis. Cogn Comput. 2017;9(6):843–51.

    Article  Google Scholar 

  39. Cambria E, Hussain A, Havasi C, Eckl C. Sentic computing: Exploitation of common sense for the development of emotion-sensitive systems. In A. Esposito, N. Campbell, C. Vogel, A. Hussain, and A. Nijholt, editors, Development of Multimodal Interfaces: Active Listening and Synchrony, Second COST 2102 International Training School, Dublin, Ireland, March 23-27, 2009, Revised Selected Papers, volume 5967 of Lecture Notes in Computer Science. p. 148–156. Springer, 2009. https://doi.org/10.1007/978-3-642-12397-9\_12

  40. Hutto CJ, Gilbert E. VADER: A parsimonious rule-based model for sentiment analysis of social media text. In E. Adar, P. Resnick, M. D. Choudhury, B. Hogan, and A. H. Oh, editors, Proceedings of the Eighth International Conference on Weblogs and Social Media, ICWSM 2014, Ann Arbor, Michigan, USA, June 1-4, 2014. The AAAI Press, 2014.

  41. Majumder N, Poria S, Peng H, Chhaya N, Cambria E, Gelbukh A. Sentiment and sarcasm classification with multitask learning. IEEE Intell Syst. 2019;34(3):38–43.

    Article  Google Scholar 

  42. Qureshi SA, Saha S, Hasanuzzaman M, Dias G. Multitask representation learning for multimodal estimation of depression level. IEEE Intell Syst. 2019;34(5):45–52.

    Article  Google Scholar 

  43. Pennington J, Socher R, Manning C. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014. p. 1532–1543.

  44. Cho K, van Merrienboer B, Bahdanau D, Bengio Y. On the properties of neural machine translation: Encoder-decoder approaches. In D. Wu, M. Carpuat, X. Carreras, and E. M. Vecchi, editors, Proceedings of SSST@EMNLP 2014, Eighth Workshop on Syntax, Semantics and Structure in Statistical Translation, Doha, Qatar, 25 October 2014, pages 103–111. Association for Computational Linguistics, 2014.

  45. Bahdanau D, Cho K, Bengio Y. Neural machine translation by jointly learning to align and translate. arXiv preprint 2014. arXiv:1409.0473

  46. Sukhbaatar S, Szlam A, Weston J, Fergus R. End-to-end memory networks. In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada. 2015. p. 2440–2448.

  47. Caruana R. Multitask learning. Mach Learn. 1997;28(1):41–75.

    Article  MathSciNet  Google Scholar 

  48. Goldberg Y. Neural network methods for natural language processing. Synthesis Lectures on Human Language Technologies. 2017;10(1):1–309.

    Article  Google Scholar 

  49. Cambria E, Fu J, Bisio F, Poria S. Affectivespace 2: Enabling affective intuition for concept-level sentiment analysis. In B. Bonet and S. Koenig, editors, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, pages 508–514. AAAI Press, 2015. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9911

  50. Bingham E, Mannila H. Random projection in dimensionality reduction: applications to image and text data. In Proceedings of the Seventh ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 2001. p. 245–250.

  51. Srivastava N, Hinton G, Krizhevsky A, Sutskever I, Salakhutdinov R. Dropout: a simple way to prevent neural networks from overfitting. J Mach Learn Res. 2014;15(1):1929–58.

    MathSciNet  MATH  Google Scholar 

  52. Kingma DP, Ba J. Adam: A method for stochastic optimization. In Y. Bengio and Y. LeCun, editors, 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015.

  53. Chollet F, et al. keras, 2015.

  54. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, et al. Scikit-learn: Machine learning in python. J Mach Learn Res. 2011;12:2825–30.

    MathSciNet  MATH  Google Scholar 

  55. Lai S, Xu L, Liu K, Zhao J. Recurrent convolutional neural networks for text classification. In B. Bonet and S. Koenig, editors, Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, p. 2267–2273. AAAI Press, 2015. http://www.aaai.org/ocs/index.php/AAAI/AAAI15/paper/view/9745

  56. Liu P, Qiu X, Huang X. Adversarial multi-task learning for text classification. In R. Barzilay and M. Kan, editors, Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, p. 1–10. Association for Computational Linguistics, 2017. https://doi.org/10.18653/v1/P17-1001

  57. LeCun Y, Bottou L, Orr GB, Müller K. Efficient backprop. In G. Montavon, G. B. Orr, and K. Müller, editors, Neural Networks: Tricks of the Trade - Second Edition, volume 7700 of Lecture Notes in Computer Science, p. 9–48. Springer, 2012. https://doi.org/10.1007/978-3-642-35289-8_3

  58. Akhtar MS, Kumar A, Ghosal D, Ekbal A, Bhattacharyya P. A multilayer perceptron based ensemble technique for fine-grained financial sentiment analysis. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 2017. p 540–546.

  59. Glorot X, Bordes A, Bengio Y. Deep sparse rectifier neural networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics. 2011. p 315–323.

Download references

Acknowledgements

Dr. Sriparna Saha gratefully acknowledges the Young Faculty Research Fellowship (YFRF) Award, supported by Visvesvaraya Ph.D. scheme for Electronics and IT, Ministry of Electronics and Information Technology (MeitY), Government of India, being implemented by Digital India Corporation (formerly Media Lab Asia) for carrying out this research.

Funding

This research has not been funded by any company or organization

Author information

Authors and Affiliations

  1. Department of Computer Science and Engineering, Indian Institute of Technology, Patna, India

    Apoorva Singh & Sriparna Saha

  2. Cork Institute of Technology Dublin, Rossa Ave, Bishopstown, Cork, T12 P928, Ireland

    Md. Hasanuzzaman

  3. Accenture Pvt. Ltd, Bengaluru, India

    Kuntal Dey

Authors
  1. Apoorva Singh
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sriparna Saha
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Md. Hasanuzzaman
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Kuntal Dey
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sriparna Saha.

Ethics declarations

Conflicts of Interest

The authors declare that they have no conflict of interest

Ethical Approval

This article does not contain any studies with human participants or animals performed by any of the authors

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Singh, A., Saha, S., Hasanuzzaman, M. et al. Multitask Learning for Complaint Identification and Sentiment Analysis. Cogn Comput 14, 212–227 (2022). https://doi.org/10.1007/s12559-021-09844-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12559-021-09844-7

Keywords

Search

Navigation