Abstract
Two sets of linguistic features are developed: The first one to estimate if a single step in a dialogue between a human being and a machine is successful or not. The second set to classify dialogues as a whole. The features are based on Part-of-Speech-Labels (POS), word statistics and properties of turns and dialogues. Experiments were carried out on the SympaFly corpus, data from a real application in the flight booking domain. A single dialogue step could be classified with an accuracy of 83% (class-wise averaged recognition rate). The recognition rate for whole dialogues was 85%.
This is a preview of subscription content, log in via an institution to check access.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Batliner, A., Hacker, C., Steidl, S., Nöth, E., Haas, J.: User States, User Strategies, and System Performance: How to Match the One with the Other. In: Proc. of ISCA-EHSD 2003, an ISCA Tutorial and Research Workshop on Error Handling in Spoken Dialogue Systems, Château d’Oex, pp. 5–10 (2003)
Hirschberg, J., Litman, D., Swerts, M.: Prosodic Cues to Recognition Errors. In: Proc. of ASRU 1999, Keystone, pp. 349–352 (1999)
Levow, G.-A.: Characterizing and Recognizing Spoken Corrections in Human-Computer Dialogue. In: Proc. of COLING/ACL 1998, Montréal, vol. 1, pp. 736–742 (1998)
Ruff, C.: Bestimmung des Dialog(schritt)erfolgs mit linguistischen Merkmalen. Studienarbeit, Lehrstuhl für Mustererkennung, Universität Erlangen-Nürnberg (2004)
Stemmer, G., Steidl, S., Nöth, E., Niemann, H., Batliner, A.: Comparison and Combination of Confidence Measures. In: Sojka, P., et al. (eds.) TSD 2002. LNCS (LNAI), vol. 2448, pp. 181–188. Springer, Heidelberg (2002)
Walker, M.A., Langkilde, I., Wright, J., Gorin, A., Litman, D.: Learning to Predict Problematic Situations in a Spoken Dialogue System: Experiments with How I Help You? In: Proc. of NAACL 2000, Seattle, pp. 210–217 (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2004 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Steidl, S., Hacker, C., Ruff, C., Batliner, A., Nöth, E., Haas, J. (2004). Looking at the Last Two Turns, I’d Say This Dialogue Is Doomed – Measuring Dialogue Success. In: Sojka, P., Kopeček, I., Pala, K. (eds) Text, Speech and Dialogue. TSD 2004. Lecture Notes in Computer Science(), vol 3206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30120-2_79
Download citation
DOI: https://doi.org/10.1007/978-3-540-30120-2_79
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-23049-6
Online ISBN: 978-3-540-30120-2
eBook Packages: Springer Book Archive