Abstract
The rapid growth of intelligent chatbots as conversational agents with the assistance of artificial intelligence has recently attracted much research attention. The major role of a chatbot is to generate appropriate responses to the user, however sometimes the chatbot fails to understand the user’s meaning. Therefore, detecting inappropriate responses from a chatbot is a critical issue. Several studies based on annotated datasets have investigated the problem of inappropriate responses from a chatbots perspective without considering the user’s perspective. Understanding the context of the conversation is an important point in determining whether a response is appropriate or inappropriate. Sentiment analysis is a natural language processing task that supports mining in user behavior. Therefore, we propose an intelligent framework that combines automated sentiment scoring and a word embedding model to detect the quality of chatbot responses considering the end-user’s point of view. We find our model achieves higher quality results than logistic regression.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bartl, A., Spanakis, G.: A retrieval-based dialogue system utilizing utterance and context embeddings. In: 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA), pp. 1120–1125. IEEE (2017)
Tripathy, A., Agrawal, A., Rath, S.K.: Classification of sentiment reviews using n-gram machine learning approach. Expert Syst. Appl. 57, 117–126 (2016)
Hastie, T., Tibshirani, R., Friedman, J.: Unsupervised learning. In: Hastie, T., Tibshirani, R., Friedman, J. (eds.) The Elements of Statistical Learning, pp. 485–585. Springer, New York (2009)
Shang, L., Lu, Z., Li, H.: Neural responding machine for short-text conversation. arXiv preprint arXiv:1503.02364 (2015)
Sugiyama, H.: Empirical feature analysis for dialogue breakdown detection. Comput. Speech Lang. 54, 140–150 (2019)
Takayama, J., Nomoto, E., Arase, Y.: Dialogue breakdown detection considering annotation biases. In: Proceedings of the Dialog System Technology Challenge 6 Workshop (DSTC6) (2017)
Xie, Z., Ling, G.: Dialogue breakdown detection using hierarchical bi-directional LSTMs. In: Proceedings of the Dialog System Technology Challenges Workshop (DSTC6) (2017)
Zeng, Z., Song, R., Lin, P., Sakai, T.: Attitude detection for one-round conversation: jointly extracting target-polarity pairs. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, pp. 285–293. ACM (2019)
Lee, C.W., Wang, Y.S., Hsu, T.Y., Chen, K.Y., Lee, H.Y., Lee, L.S.: Scalable sentiment for sequence-to-sequence chatbot response with performance analysis. In: 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6164–6168. IEEE (2018)
Ireland, D., Hassanzadeh, H., Tran, S.N.: Sentimental analysis for AIML-based e-health conversational agents. In: International Conference on Neural Information Processing, pp. 41–51. Springer (2018)
Rinaldi, A., Oseguera, O., Tuazon, J., Cruz, A.C.: End-to-end dialogue with sentiment analysis features. In: International Conference on Human-Computer Interaction, pp. 480–487. Springer (2017)
Hutto, C.J., Gilbert, E.: VADER: a parsimonious rule-based model for sentiment analysis of social media text. In: Eighth International AAAI Conference on Weblogs and Social Media (2014)
Poria, S., Hazarika, D., Majumder, N., Naik, G., Cambria, E., Mihalcea, R.: Meld: a multimodal multi-party dataset for emotion recognition in conversations. arXiv preprint arXiv:1810.02508 (2018)
Danescu-Niculescu-Mizil, C., Lee, L.: Chameleons in imagined conversations: a new approach to understanding coordination of linguistic style in dialogs. In: Proceedings of the 2nd Workshop on Cognitive Modeling and Computational Linguistics, pp. 76–87. Association for Computational Linguistics (2011)
Raschka, S.: Python Machine Learning. Packt Publishing Ltd., Birmingham (2015)
Müller, A.C., Guido, S., et al.: Introduction to Machine Learning with Python: A Guide for Data Scientists. O’Reilly Media Inc., Sebastopol (2016)
Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inf. Process. Manag. 45(4), 427–437 (2009)
Raza, M., Hussain, F.K., Hussain, O.K., Zhao, M., ur Rehman, Z.: A comparative analysis of machine learning models for quality pillar assessment of SaaS services by multi-class text classification of users’ reviews. Future Gen. Comput. Syst. 101, 341–371 (2019)
Yin, Z., Shen, Y.: On the dimensionality of word embedding. CoRR abs/1812.04224 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Almansor, E.H., Hussain, F.K. (2020). Modeling the Chatbot Quality of Services (CQoS) Using Word Embedding to Intelligently Detect Inappropriate Responses. In: Barolli, L., Amato, F., Moscato, F., Enokido, T., Takizawa, M. (eds) Advanced Information Networking and Applications. AINA 2020. Advances in Intelligent Systems and Computing, vol 1151. Springer, Cham. https://doi.org/10.1007/978-3-030-44041-1_6
Download citation
DOI: https://doi.org/10.1007/978-3-030-44041-1_6
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-44040-4
Online ISBN: 978-3-030-44041-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)