Abstract
The spoken dialogue systems (SDSs), which are designed to replace employees in different services, need some indicators, which show what happened in the ongoing dialogue and what the next step in system’s behaviour should be. Thus, some indicators for the SDSs come from the field of the call centre’s quality evaluation. In turn, some metrics like Interaction Quality (IQ), which was designed for human-computer spoken interaction, can be applied to human-human conversations. Such experience might be used for both call centres and SDSs for service quality improvement. This paper provides the results of IQ modelling for human-human task-oriented conversation with several classification algorithms.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Schmitt, A., Schatz, B., Minker, W.: Modeling and predicting quality in spoken human-computer interaction. In: Proceedings of the SIGDIAL 2011 Conference. Association for Computational Linguistics, pp. 173–184 (2011)
Schmitt, A., Ultes, S., Minker, W.: A parameterized and annotated corpus of the CMU let’s go bus information system. In: International Conference on Language Resources and Evaluation (LREC) (2012)
Schmitt, A., Ultes, S.: Interaction Quality: Assessing the quality of ongoing spoken dialog interaction by experts – And how it relates to user satisfaction. Speech Commun. 74, 12–36 (2015)
Devillers, L., Vidrascu, L.: Real-life emotions detection with lexical and paralinguistic cues on human-human call center dialogs. Proc. Interspeech 2006, 801–804 (2006)
Siegert, I., Ohnemus, K.: A new dataset of telephone-based human-human call-center interaction with emotional evaluation. In: Proceedings of the 1st International Symposium on Companion-Technology (ISCT 2015), pp. 143–148 (2015)
Rafaeli, A., Ziklik, L., Doucet, L.: The impact of call center employees’ customer orientation behaviors on service quality. J. Serv. Res. 10(3), 239–255 (2008)
Pallotta, V., Delmonte, R.: Interaction mining: the new frontier of customer interaction analytics. In: Lai, C., Semeraro, G., Vargiu, E. (eds.) New Challenges in Distributed Inf. Filtering and Retrieval. SCI, vol. 439, pp. 91–111. Springer, Heidelberg (2013)
Park, Y., Gates, S.C.: Towards real-time measurement of customer satisfaction using automatically generated call transcripts. In: Proceedings of the 18th ACM Conference on Information and Knowledge Management, pp. 1387–1396 (2009)
Walker, M., Kamm, C., Boland, J.: Developing and testing general models of spoken dialogue system performance. In: Proceedings of the 2nd International Conference on Language Resources and Evaluation (LREC), pp. 189–196 (2000)
Higashinaka, R., Minami, Y., Dohsaks, K., Meguro, T.: Modeling user satisfaction transitions in dialogues from overall ratings. In: Proceedings of SIGDIAL 2010: the 11th Annual Meetings of the Special Interest Group on Discourse and Dialogue, pp. 18–27 (2010)
Spirina, A.V., Sidorov, M.Y., Sergienko, R.B., Semenkin E.S., Minker, W.: Human-human task-oriented conversations corpus for interaction quality modelling. Vestnik SibSAU, 17(1), 84–90 (2016) (In Russian). http://vestnik.sibsau.ru/images/vestnik/ves%2017%20num%201.pdf
RapidMiner. http://rapidminer.com/
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., Witten, I.H.: The WEKA data mining software: an update. SIGKDD Explor. 11(1), 37–57 (2009)
John, G.H., Langley, P.: Estimating continuous distributions in bayesian classifiers. In: Eleventh Conference on Uncertainty in Artificial Intelligence, San Mateo, pp. 338–345 (1995)
Witten, I.H., Frank, E., Hall, M.A.: Data mining: practical machine learning tools and techniques, 3rd edn. Morgan Kaufmann, USA (2011)
le Cessie, S., van Houwelingen, J.C.: Ridge estimators in logistic regression. Appl. Stat. 41(1), 191–201 (1992)
Cristianini, N., Shawe-Taylor, J.: An introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press, Cambridge (2000)
Vapnik, V.N.: The nature of statistical learning theory. Springer-Verlag New York, Inc., New York (1995)
Platt, J.: Sequential Minimal Optimisation: A Fast Algorithm for Training Support Vector Machines. TechReport MSR-TR-98-14, Microsoft Research (1998)
Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. Advances in information retrieval. Springer, Heidelberg (2005)
Abdi, H., Williams, L.J.: Principal component analysis. Computational Statistics 2, 433–459, John Wiley & Sons, Inc. (2010)
Rosenberg, A.: Classifying skewed data: importance to optimize average recall. Proc. Interspeech 2012, 2242–2245 (2012)
Bailey, R.A.: Design of comparative experiments. Cambridge University Press, New York (2008)
Kennedy, J.J., Bush, A.J.: An introduction to the design and analysis of experiments in behavioral research. University Press of America, New York (1985)
R Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2015). http://www.r-project.org/
Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52(3–4), 591–611 (1965)
Spearman, C.: The proof and measurement of association between two things. Am. J. Psychol. 15(1), 72–101 (1904)
Acknowledgements
The work presented in this paper was partly supported by the DAAD (German Academic Exchange Service) within the different programmes.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Spirina, A., Vaskovskaia, O., Sidorov, M., Schmitt, A. (2016). Interaction Quality as a Human-Human Task-Oriented Conversation Performance. In: Ronzhin, A., Potapova, R., Németh, G. (eds) Speech and Computer. SPECOM 2016. Lecture Notes in Computer Science(), vol 9811. Springer, Cham. https://doi.org/10.1007/978-3-319-43958-7_48
Download citation
DOI: https://doi.org/10.1007/978-3-319-43958-7_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-43957-0
Online ISBN: 978-3-319-43958-7
eBook Packages: Computer ScienceComputer Science (R0)