Abstract
In this paper, we reduce the rescoring problem in a spoken dialogue understanding task to a classification problem, by using the semantic error rate as the reranking target value. The classifiers we consider here are trained with linguistically motivated features. We present comparative experimental evaluation results of four supervised machine learning methods: Support Vector Machines, Weighted K-Nearest Neighbors, Naïve Bayes and Conditional Inference Trees. We provide a quantitative evaluation of learning and generalization during the classification supervised training, using cross validation and ROC analysis procedures. The reranking is derived using the posterior knowledge given by the classification algorithms.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wessel, F., Schlüter, R., Macherey, K., Ney, H.: Confidence Measures for Large Vocabulary Continuous Speech Recognition. IEEE Transactions on Speech and Audio Processing 9 (2001)
Gandrabur, S., Foster, G., Lapalme, G.: Confidence Estimation for NLP Applications. ACM Transactions on Speech and Language Processing 3, 1–29 (2006)
Torres, F., Hurtado, L., García, F., Sanchis, E., Segarra, E.: Error handling in a stochastic dialog system through confidence measures. Speech Communication 45, 211–229 (2005)
Mangu, L., Brill, E., Stolcke, A.: Finding consensus in speech recognition: word error minimization and other applications of confusion networks. Computer Speech and Language 14, 373–400 (2000)
Stephenson, T.A., Doss, M.M., Bourlard, H.: Speech Recognition With Auxiliary Information. IEEE Transactions on Speech and Audio Processing 12 (2004)
Dinarelli, M., Moschitti, A., Riccardi, G.: Re-Ranking Models For Spoken Language Understanding. In: Proceedings of the 12th Conference of the European Chapter of the ACL, Athens, Greece, pp. 202–210 (2009)
McNeilly, W.P., Kahn, J.G., Hillard, D.L., Ostendorf, M.: Parse Structure and Segmentation for Improving Speech Recognition. In: Proceedings of the IEEE/ACL Workshop on Spoken Language Technology, Aruba (2006)
Chotimongkol, A., Rudnicky, A.I.: N-best Speech Hypotheses Reordering Using Linear Regression. In: Proceedings of the Seventh European Conference on Speech Communication and Technology (EuroSpeech), Aalborg, Denmark, pp. 1829–1832 (2001)
Walker, M., Wright, J., Langkilde, I.: Using Natural Language Processing and Discourse Features to Identify Understanding Errors in a Spoken Dialogue System. In: Proceedings of the 17th International Conference on Machine Learning (2000)
Brill, E., Florian, R., Henderson, J.C., Mangu, L.: Beyond N-Grams: Can Linguistic Sophistication Improve Language Modeling? In: Proceedings of the International Conference On Computational Linguistics (COLING), Montreal, Canada, pp. 186–190 (1998)
Jonson, R.: Dialogue Context-Based Re-ranking of ASR Hypotheses. In: Proceedings of the Spoken Language Technology Workshop, Aruba, pp. 174–177 (2006)
Bohus, D., Rudnicky, A.: Constructing Accurate Beliefs in Spoken Dialog Systems. In: IEEE SPS 2005 Automatic Speech Recognition and Understanding Workshop, San Juan, Puerto Rico (2005a)
Bohus, D., Rudnicky, A.I.: Error Handling in the RavenClaw Dialog Management Framework. In: Proceedings of the Conference on Human Language Technology and Empirical Methods in Natural Language Processing (HLT/EMNLP), Association for Computational Linguistics, Morristown, NJ, USA, Vancouver, British Columbia, Canada, pp. 225–232 (2005)
Bohus, D., Rudnicky, A.: A “K Hypotheses + Other” Belief Updating Model. In: AAAI Workshop on Statistical and Empirical Approaches for Spoken Dialogue systems, Boston, USA (2006)
Williams, J.: Exploiting the ASR N-Best by tracking multiple dialog state hypotheses. In: Interspeech, Brisbane, Australia (2008)
Gabsdil, M.: Classifying Recognition Results for Spoken Dialogue Systems. In: Proceedings of the 41st Annual Meeting on Association for Computational Linguistics, Sapporo, Japan, pp. 23–30 (2003)
Gabsdil, M., Lemon, O.: Combining Acoustic and Pragmatic Features to Predict Recognition Performance in Spoken Dialogue Systems. In: Proceedings of the 42nd Meeting of the Association for Computational Linguistics (ACL), Barcelona, Spain, pp. 343–350 (2004)
Higashinaka, R., Nakano, M., Aikawa, K.: Corpus-based discourse understanding in spoken dialogue systems. In: Proceedings the 41st Annual Meeting on Association for Computational Linguistics (2006)
Georgescul, M., Rayner, M., Bouillon, P., Tsourakis, N.: Discriminative Learning Using Linguistic Features to Rescore N-best Speech Hypotheses. In: Proceedings of the IEEE Workshop on Spoken Language Technology, Goa, India (2008)
Raymond, C., Béchet, F., Camelin, N., De Mori, R., Damnati, G.: Sequential decision strategies for machine interpretation of speech. IEEE Transactions on Audio, Speech and Language Processing 15 (2007)
Dinarelli, M., Moschitti, A., Riccardi, G.: Re-ranking models based-on small training data for spoken language understanding. In: Proceedings of the Conference on Empirical Methods in Natural Language Processing, Singapore, pp. 1076–1085 (2009)
Tetreaulta, J.R., Litman, D.J.: A Reinforcement Learning approach to evaluating state representations in spoken dialogue systems. Speech Communication 50, 683–696 (2008)
Thomson, B., Yu, K., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Young, S.: Evaluating semantic-level confidence scores with multiple hypotheses. In: Proceedings of Interspeech, Brisbane, Australia (2008)
Joachims, T.: Optimizing Search Engines Using Clickthrough Data. In: Proceedings of the 8th ACM International Conference on Knowledge Discovery and Data Mining (KDD), Edmonton, Alberta, Canada (2002)
Tsourakis, N., Georgescul, M., Bouillon, P., Rayner, M.: Building Mobile Spoken Dialogue Applications Using Regulus. In: Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC), Marrakech, Morocco (2008)
Rayner, M., Hockey, B.A., Bouillon, P.: Putting Linguistics into Speech Recognition. Center for the Study of Language and Information (2006)
Clarkson, P., Rosenfeld, R.: Statistical Language Modeling Using the CMU Cambridge Toolkit. In: Proceedings of the ESCA Eurospeech (1997)
Hechenbichler, K., Schliep, K.: Weighted k-Nearest-Neighbor Techniques and Ordinal Classification. Institut für Statistik, Ludwig-Maximilians University München, pp. 1–16 (2004)
Hothorn, T., Hornik, K., Zeileis, A.: Unbiased Recursive Partitioning: A Conditional Inference Framework. Journal of Computational and Graphical Statistics 15, 651–674 (2006)
Platt, J.C.: Probabilistic outputs for support vector machines and comparison to regularized likelihood methods. MIT Press, Cambridge (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Georgescul, M., Rayner, M., Bouillon, P. (2010). Spoken Language Understanding via Supervised Learning and Linguistically Motivated Features. In: Hopfe, C.J., Rezgui, Y., Métais, E., Preece, A., Li, H. (eds) Natural Language Processing and Information Systems. NLDB 2010. Lecture Notes in Computer Science, vol 6177. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-13881-2_12
Download citation
DOI: https://doi.org/10.1007/978-3-642-13881-2_12
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-13880-5
Online ISBN: 978-3-642-13881-2
eBook Packages: Computer ScienceComputer Science (R0)