Abstract
For spoken dialog systems, PARADISE [Walker et al. 1997] provides a framework to train a user satisfaction prediction model on given data. The approach weights and sums interaction parameters to predict a satisfaction metric calculated from a questionnaire. In this paper, we try to tackle a major problem of these models, namely their weak generalizability. We show, that the weights associated with interaction parameters in the model change in dependence of the system’s major problems by examining correlations under different quantities of understanding errors in the dialogs.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Walker, M.A., Litman, D.J., Kamm, C.A., Abella, A.: PARADISE: A Framework for Evaluating Spoken Dialog Agents. In: Proc. Of the ACL/EACL 35th Ann. Meeting of the Assoc. for Computational Linguistics, pp. 271–280 (1997)
Nielsen, J.: Usability Engineering. Morgan Kaufmann, Amsterdam (1993)
Hajdinjak, M., Mihelič, F.: The PARADISE Evaluation Framework: Issues and Findings. Computational Linguistics 32, 263–272 (2006)
Trutnev, A., Ronzenknop, A., Rajman, M.: Speech Recognition Simulation and its Application for Wizard-of-Oz Experiments. In: Proc. of 4th LREC, pp. 611–614 (2004)
Möller, S., Smeele, P., Boland, H., Krebber, J.: Evaluating Spoken Dialog Systems According to De-facto Standards: A Case Study. Computer Speech and Language 21, 26–53 (2007)
ITU-T Suppl. 24 to P-Series Rec. Parameters Describing the Interaction with Spoken Dialogue Systems. International Telecommunication Union, Geneva (2005)
Hone, K.S., Graham, R.: Towards a Tool for the Subjective Assessment of Speech System Interfaces (SASSI). Natural Lang. Engineering 6, 287–303 (2000)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Engelbrecht, KP., Kühnel, C., Möller, S. (2008). Weighting the Coefficients in PARADISE Models to Increase Their Generalizability. In: André, E., Dybkjær, L., Minker, W., Neumann, H., Pieraccini, R., Weber, M. (eds) Perception in Multimodal Dialogue Systems. PIT 2008. Lecture Notes in Computer Science(), vol 5078. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69369-7_34
Download citation
DOI: https://doi.org/10.1007/978-3-540-69369-7_34
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69368-0
Online ISBN: 978-3-540-69369-7
eBook Packages: Computer ScienceComputer Science (R0)