Abstract
Estimation of the dialogue quality, especially the quality of interaction, is an essential part for improving the quality of spoken dialogue systems (SDSs) or call centres. The Interaction Quality (IQ) metric is one of such approaches. Originally, it was designed for SDSs to estimate an ongoing human-computer spoken interaction (HCSI). Due to a similarity between task-oriented human-human conversation (HHC) and HCSI, this approach was adapted to HHC. As for HCSI, for HHC the IQ model is based on features from three interaction parameter levels: an exchange, a window, and a dialogue level. We determine the significance of different levels for IQ modelling for HHC. Moreover, for the window level we try to find an optimal window size. Our study was aimed to simplify the IQ model for HHC, as well as to find differences and similarities between IQ models for HHC and HCSI.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Schmitt, A., Schatz, B., Minker, W.: Modeling and predicting quality in spoken human-computer interaction. In: Proceedings of the SIGDIAL 2011 Conference, Association for Computational Linguistics, pp. 173–184 (2011)
Schmitt, A., Ultes, S., Minker, W.: A parameterized and annotated corpus of the CMU lets go bus information system. In: International Conference on Language Resources and Evaluation (LREC), pp. 3369–3373 (2012)
Spirina, A., Sidorov, M., Sergienko, R., Schmitt, A.: First experiments on interaction quality modelling for human-human conversation. In: Proceedings of the 13th International Conference on Informatics in Control, Automation and Robotics (ICINCO), vol. 2, pp. 374–380 (2016)
Spirina, A.V., Sidorov, M.Y., Sergienko, R.B., Semenkin, E.S., Minker, W.: Human-human task-oriented conversations corpus for interaction quality modelling. Vestnik SibSAU 17(1), 84–90 (2016)
Ultes, S., Schmitt, A., Minker, W.: Analysis of temporal features for interaction quality estimation. In: Jokinen, K., Wilcock, G. (eds.) Dialogues with Social Robots. LNEE, vol. 999, pp. 367–379. Springer, Singapore (2017). doi:10.1007/978-981-10-2585-3_30
Schmitt, A., Ultes, S.: Interaction quality: assessing the quality of ongoing spoken dialog interaction by experts and how it relates to user satisfaction. Speech Commun. 74, 12–36 (2015)
Rosenberg, A.: Classifying skewed data: importance to optimize average recall. In: Proceedings of INTERSPEECH 2012, pp. 2242–2245 (2012)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Measur. 20, 37–46 (1960)
Cohen, J.: Weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. Psychol. Bull. 70(4), 213–220 (1968)
Spearman, C.: The proof and measurement of association between two things. Am. J. Psychol. 15(1), 72–101 (1904)
Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in opensmile, the munich open-source multimedia feature extractor. In: Proceedings of ACM Multimedia (MM), pp. 835–838 (2013)
Schuller, B., Steidl, S., Batliner, A.: The interspeech 2009 emotion challenge. In: Proceedings of INTERSPEECH 2009, pp. 312–315 (2009)
Sidorov, M., Brester, C., Schmitt, A.: Contemporary stochastic feature selection algorithms for speech-based emotion recognition. In: Proceedings of INTERSPEECH 2015, pp. 2699–2703 (2015)
Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutmann, P., Witten, I.H.: The weka data mining software: an update. SIGKDD Explor. 11(1) (2009)
John, G.H., Langley, P.: Estimating continuous distribution in bayesian classifiers. In: Eleventh Conference on Uncertainty in Artificial Intelligence, pp. 338–345 (1995)
Witten, I.H., Frank, E., Hall, M.A.: Data mining: practical machine learning tools and techniques. Morgan Kaufmann, USA (2011)
le Cessie, S., Houwelingen, J.C.: Ridge estimators in logistic regression. Appl. Stat. 41(1), 191–201 (1992)
Cristianini, N., Shawe-Taylor, J.: An Introduction to Support Vector Machines and Other Kernel-based Learning Methods. Cambridge University Press (200)
Vapnik, V.N.: The Nature of Statistical Learning Theory. Springer, New York (1995)
Platt, J.: Fast training of support vector machines using sequential minimal optimization. Adv. Kernel Methods Support Vector Learn. 3 (1999)
Goutte, C., Gaussier, E.: A probabilistic interpretation of precision, recall and f-score, with implication for evaluation. Advances in Information Retrieval, pp. 345–359 (2005)
Abdi, H., Williams, L.: Principal component analysis. WIREs Comput. Stat. 2, 433–459 (2010)
Spirina, A., Vaskovskaia, O., Sidorov, M., Schmitt, A.: Interaction quality as a human-human task-oriented conversation performance. In: Proceedings of the 18th International Conference on Speech and Computer (SPECOM 2016), pp. 403–410 (2016)
Bailey, R.A.: Design of Comparative Experiments. Cambridge University Press (2008)
Kennedy, J.J., Bush, A.J.: An Introduction to the Design and Analysis of Experiments in Behavioural Research. University Press of America (1985)
Fay, M.P., Proschan, M.A.: Wilcoxon-mann-whitney or t-test? on assumptions for hypothesis tests and multiple interpretations of decision rules. Stat. Surv. 4, 1–39 (2010)
Acknowledgments
The work presented in this paper was partially supported by the DAAD (German Academic Exchange Service), the Ministry of Education and Science of Russian Federation within project 28.697.2016/2.2, and the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” which is funded by the German Research Foundation (DFG).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2017 Springer International Publishing AG
About this paper
Cite this paper
Spirina, A., Vaskovskaia, O., Karaseva, T., Skorokhod, A., Polonskaia, I., Sidorov, M. (2017). Analysis of Interaction Parameter Levels in Interaction Quality Modelling for Human-Human Conversation. In: Karpov, A., Potapova, R., Mporas, I. (eds) Speech and Computer. SPECOM 2017. Lecture Notes in Computer Science(), vol 10458. Springer, Cham. https://doi.org/10.1007/978-3-319-66429-3_12
Download citation
DOI: https://doi.org/10.1007/978-3-319-66429-3_12
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-66428-6
Online ISBN: 978-3-319-66429-3
eBook Packages: Computer ScienceComputer Science (R0)