Mayur Patidar
Gautam Shroff
ABSTRACT
Helpdesk is a key component of any large IT organization, where users can log a ticket about any issue they face related to IT infrastructure, administrative services, human resource services, etc. Normally, users have to assign appropriate set of labels to a ticket so that it could be routed to right domain expert who can help resolve the issue. In practice, the number of labels are very large and organized in form of a tree. It is non-trivial to describe the issue completely and attach appropriate labels unless one knows the cause of the problem and the related labels. Sometimes domain experts discuss the issue with the users and change the ticket labels accordingly, without modifying the ticket description. This results in inconsistent and badly labeled data, making it hard for supervised algorithms to learn from. In this paper, we propose a novel approach of creating a conversational helpdesk system, which will ask relevant questions to the user, for identification of the right category and will then raise a ticket on users' behalf. We use attention based seq2seq model to assign the hierarchical categories to tickets. We use a slot filling model to help us decide what questions to ask to the user, if the top-k model predictions are not consistent. We also present a novel approach to generate training data for the slot filling model automatically based on attention in the hierarchical classification model. We demonstrate via a simulated user that the proposed approach can give us a significant gain in accuracy on ticket-data without asking too many questions to users. Finally, we also show that our seq2seq model is as versatile as other approaches on publicly available datasets, as state of the art approaches.
- Mucahit Altintas and Cuneyd Tantug. 2014. Machine Learning Based Ticket Classification in Issue Tracking Systems. In Proceeding of the International Conference on Artificial Intelligence and Computer Science (AICS).Google Scholar
- Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural Machine Translation by Jointly Learning to Align and Translate. CoRR abs/1409.0473 (2014).Google Scholar
- Jayme Garcia Arnal Barbedo and Amauri Lopes. 2007. Automatic Genre Classification of Musical Signals. EURASIP Journal on Advances in Signal Processing 2007, 1 (Jan. 2007). Google ScholarDigital Library
- Zafer Barutcuoglu and Christopher DeCoro. 2006. Hierarchical Shape Classification Using Bayesian Aggregation. In Proceedings of the IEEE International Conference on Shape Modeling and Applications 2006 (SMI '06). Google ScholarDigital Library
- Daniel Beneker and Carsten Gips. 2017. Using Clustering for Categorization of Support Tickets. In LWDA.Google Scholar
- Juan José Burred and Alexander Lerch. 2003. A Hierarchical Approach to Automatic Musical Genre Classification. In Proceedings of the 6th International Conference on Digital Audio Effects (DAFx-03).Google Scholar
- Anveshi Charuvaka and Huzefa Rangwala. 2015. HierCost: Improving Large Scale Hierarchical Classification with Cost Sensitive Learning. In Proceedings of the 2015th European Conference on Machine Learning and Knowledge Discovery in Databases - Volume Part I (ECMLPKDD'15). Springer, Switzerland. Google ScholarDigital Library
- Kyunghyun Cho, Bart van Merrienboer, et al. 2014. Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. (2014).Google Scholar
- Stephen D'Alessio et al. 2000. The Effect of Using Hierarchical Classifiers in Text Categorization. In Content-Based Multimedia Information Access - Volume 1 (RIAO '00). Google ScholarDigital Library
- Ofer Dekel, Joseph Keshet, and Yoram Singer. 2004. Large Margin Hierarchical Classification. In Proceedings of the Twenty-first International Conference on Machine Learning (ICML '04). ACM. Google ScholarDigital Library
- Y. Diao, H. Jamjoom, and D. Loewenstern. 2009. Rule-Based Problem Classification in IT Service Management. In 2009 IEEE International Conference on Cloud Computing. Google ScholarDigital Library
- Alex A. Freitas. 2007. A Tutorial on Hierarchical Classification with Applications in Bioinformatics. In In: D. Taniar (Ed.) Research and Trends in Data Mining Technologies and Applications, Idea Group, 2007. 175--208.Google ScholarCross Ref
- Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Comput. (1997).Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. CoRR (2014).Google Scholar
- Daphne Koller and Mehran Sahami. 1997. Hierarchically Classifying Documents Using Very Few Words. In Proceedings of the Fourteenth International Conference on Machine Learning (ICML '97). Google ScholarDigital Library
- Kamran Kowsari, Donald E Brown, et al. 2017. HDLTex: Hierarchical Deep Learning for Text Classification. In 2017 16th IEEE International Conference on Machine Learning and Applications (ICMLA). 364--371.Google Scholar
- Taku Kudo and Yuji Matsumoto. 2001. Chunking with Support Vector Machines. In Proceedings of the Second Meeting of the North American Chapter of the Association for Computational Linguistics on Language Technologies (NAACL '01). Association for Computational Linguistics. Google ScholarDigital Library
- Jiwei Li, Will Monroe, et al. 2016. Deep Reinforcement Learning for Dialogue Generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas.Google ScholarCross Ref
- Bing Liu and Ian Lane. 2015. Recurrent Neural Network Structured Output Prediction for Spoken Language Understanding.Google Scholar
- Bing Liu and Ian Lane. 2016. Attention-Based Recurrent Neural Network Models for Joint Intent Detection and Slot Filling. CoRR abs/1609.01454 (2016).Google Scholar
- Minh-Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective Approaches to Attention-based Neural Machine Translation. CoRR abs/1508.04025 (2015).Google Scholar
- Senthil Mani, Neelamadhav Gantayat, et al. 2018. Hi, How Can I Help You?: Automating Enterprise IT Support Help Desks. CoRR abs/1711.02012 (2018).Google Scholar
- G. Mesnil, Y. Dauphin, K. Yao, et al. 2015. Using Recurrent Neural Networks for Slot Filling in Spoken Language Understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing (2015). Google ScholarDigital Library
- Tomas Mikolov, Ilya Sutskever, et al. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26. Google ScholarDigital Library
- Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. GloVe: Global Vectors for Word Representation. In Empirical Methods in Natural Language Processing (EMNLP).Google Scholar
- Christian Raymond and Giuseppe Riccardi. 2007. Generative and discriminative algorithms for spoken language understanding. In INTERSPEECH.Google Scholar
- M. Schuster and K.K. Paliwal. November 1997. Bidirectional Recurrent Neural Networks. Trans. Sig. Proc. (November 1997). Google ScholarDigital Library
- Iulian V. Serban, Alessandro Sordoni, et al. 2016. Building End-to-end Dialogue Systems Using Generative Hierarchical Neural Network Models. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI'16). AAAI Press. Google ScholarDigital Library
- Qihong Shao, Yi Chen, et al. 2008. EasyTicket: A Ticket Routing Recommendation Engine for Enterprise Problem Resolution. Proc. VLDB Endow. 1, 2 (Aug. 2008). Google ScholarDigital Library
- Qihong Shao, Yi Chen, et al. 2008. Efficient Ticket Routing by Resolution Sequence Mining. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (KDD '08). ACM. Google ScholarDigital Library
- Carlos N. Silla and Alex A. Freitas. 2011. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery (2011). Google ScholarDigital Library
- Carlos N. Silla Jr. and Alex A. Freitas. 2009. A Global-Model Naive Bayes Approach to the Hierarchical Prediction of Protein Functions. In Proceedings of the 2009 Ninth IEEE International Conference on Data Mining (ICDM '09). Google ScholarDigital Library
- Nitish Srivastava, Geoffrey E Hinton, et al. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research (2014). Google ScholarDigital Library
- Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to Sequence Learning with Neural Networks. CoRR abs/1409.3215 (2014).Google ScholarDigital Library
- Gökhan Tür, Dilek Z. Hakkani-Tür, et al. 2011. Sentence simplification for spoken language understanding. 2011 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) (2011), 5628--5631.Google ScholarCross Ref
- Oriol Vinyals and Quoc Le. 2015. A Neural Conversational Model. (06 2015).Google Scholar
- Frank Wilcoxon. 1992. Individual Comparisons by Ranking Methods. Springer New York.Google Scholar
- C. Zeng, T. Li, L. Shwartz, and G. Y. Grabarnik. 2014. Hierarchical multi-label classification over ticket data using contextual loss. In 2014 IEEE Network Operations and Management Symposium (NOMS).Google Scholar
- W. Zhou, L. Tang, et al. 2016. Resolution Recommendation for Event Tickets in Service Management. IEEE Transactions on Network and Service Management 13 (2016). Google ScholarDigital Library
- Wubai Zhou, Wei Xue, et al. 2017. STAR: A System for Ticket Analysis and Resolution. In KDD. Google ScholarDigital Library
Index Terms
- Automatic Conversational Helpdesk Solution using Seq2Seq and Slot-filling Models
Recommendations
Designing IT Service Management at Indonesia Internet Domain Names Registry Association's Helpdesk Function
ICICM '20: Proceedings of the 10th International Conference on Information Communication and ManagementIndonesia Internet Domain Names Registry Association (PANDI) hosts helpdesk function as a part of .id domain management. It is a one-stop facility for all registrants and the public to submit questions and complaints regarding .id domain names. ...
An empirical study to evaluate the impact of mindfulness on helpdesk employees
AbstractPurpose: Mindfulness is a meditation technique whose main goal involves maintaining a calm mind and training attention by focusing only on a single thing (the support) at a time; this support is usually the practitioner'...
Highlights- An industrial experiment carried out in a complex helpdesk of Accenture is presented.
Combating with extremely noisy samples in weakly supervised slot filling for automatic diagnosis
AbstractSlot filling, to extract entities for specific types of information (slot), is a vitally important modular of dialogue systems for automatic diagnosis. Doctor responses can be regarded as the weak supervision of patient queries. In this way, a ...
Comments