Abstract
We propose a method to speed up reinforcement learning of policies for spoken dialogue systems. This is achieved by combining a coarse grained abstract representation of states and actions with learning only in frequently visited states. The value of unsampled states is approximated by a linear interpolation of known states. Experiments show that the proposed method effectively optimizes dialogue strategies for frequently visited dialogue states.
Preview
Unable to display preview. Download preview PDF.
Similar content being viewed by others
References
Singh, S., Litman, D., Kearns, M., Walker, M.: Optimizing Dialogue Management with Reinforcement Learning: Experiments with the NJFun System. Journal of Artificial Intelligence Research 16, 105–133 (2002)
Gordon, G.J.: Stable function approximation in dynamic programming. In: Proceedings of the Twelfth International Conference on Machine Learning (1995)
Levin, E., Pieraccini, R.: A Stochastic Model of Human Computer Interaction for Learning Dialog Strategies. In: Proceedings of Eurospeech, Rhodos, Greece (1997)
Kaelbling, L.P., Littman, M.L., Moore, A.W.: Reinforcement Learning: A Survey. Journal of Artificial Intelligence Research 4, 237–285 (1996)
Walker, M., Fromer, J., Narayanan, S.: Learning optimal dialogue strategies: A case study of a spoken dialogue agent for email. In: Proceedings of ACL/COLING 1998 (1998)
Williams, J.D., Young, S.: Using Wizard-of-Oz Simulations to Bootstrap Reinforcement Learning Based Dialog Management Systems. In: Proceedings of the 4th SigDial Workshop on Discourse and Dialogue (2003)
Roy, N., Pineau, J., Thrun, S.: Spoken Dialog Management for Robots. In: Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics (2000)
Scheffler, K., Young, S.J.: Corpus-based dialogue simulation for automatic strategy learning and evaluation. In: Proceedings NAACL Workshop on Adaptation in Dialogue Systems, pp. 64–70 (2001)
Sutton, R.S., Barto, A.G.: Reinforcement Learning. MIT Press, Cambridge (1998)
Goddeau, D., Pineau, J.: Fast Reinforcement Learning of Dialog Strategies. In: IEEE Conference on Acoustics, Speech and Signal Processing (ICASSP), Istanbul, Turkey (2000)
Denecke, M.: Informational Characterization of Dialogue States. In: Proceedings of the 6th International Conference on Speech and Language Processing, Beijing, China (2000)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Denecke, M., Dohsaka, K., Nakano, M. (2005). Fast Reinforcement Learning of Dialogue Policies Using Stable Function Approximation. In: Su, KY., Tsujii, J., Lee, JH., Kwong, O.Y. (eds) Natural Language Processing – IJCNLP 2004. IJCNLP 2004. Lecture Notes in Computer Science(), vol 3248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-30211-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-540-30211-7_1
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-24475-2
Online ISBN: 978-3-540-30211-7
eBook Packages: Computer ScienceComputer Science (R0)