Abstract
In this work we present an architecture for Adaptive Dialogue Systems and a novel system that serves as a Museum Guide. It employs several online Reinforcement Learning (RL) techniques to achieve adaptation to the environment as well as to different users. Not many systems have been proposed that apply online RL methods and this is one of the first to fully describe an Adaptive Dialogue System with online dialogue policy learning. We evaluate our system through user simulations and compare the several implemented algorithms on a simple scenario.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Bohus, D., Rudnicky, A.I.: The RavenClaw dialog management framework: Architecture and systems. Computer Speech & Language 23(3), 332–361 (2009)
Cuayáhuitl, H., Renals, S., Lemon, O., Shimodaira, H.: Evaluation of a hierarchical reinforcement learning spoken dialogue system. Comput. Speech Lang. 24, 395–429 (2010)
Gašić, M., Jurčíček, F., Thomson, B., Yu, K., Young, S.: On-line policy optimisation of spoken dialogue systems via live interaction with human subjects. In: Automatic Speech Recognition and Understanding, Hawaii (2011)
Jurčíček, F., Thomson, B., Keizer, S., Mairesse, F., Gašić, M., Yu, K., Young, S.: Natural Belief-Critic: A Reinforcement Algorithm for Parameter Estimation in Statistical Spoken Dialogue Systems. International Speech Communication Association 7, 1–26 (2010)
Konstantopoulos, S.: An Embodied Dialogue System with Personality and Emotions. In: Proceedings of the 2010 Workshop on Companionable Dialogue Systems, ACL 2010, pp. 31–36 (2010)
Peng, J., Williams, R.: Incremental multi-step Q-Learning. Machine Learning, 283–290 (1996)
Pietquin, O., Geist, M., Chandramohan, S., Frezza-Buet, H.: Sample-Effcient Batch Reinforcement Learning for Dialogue Management Optimization. ACM Transactions on Speech and Language Processing 7(3), No. 7 (2011)
Pietquin, O., Hastie, H.: A survey on metrics for the evaluation of user simulations. The Knowledge Engineering Review (2011) (to appear)
Rieser, V., Lemon, O.: Natural Language Generation as Planning Under Uncertainty for Spoken Dialogue Systems. In: EACL 2009, pp. 683–691 (2009)
Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. The MIT Press, Cambridge (1998)
Szepesvári, C.: Algorithms for Reinforcement Learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, vol. 4(1), pp. 1–103. Morgan & Claypool Publishers (2010)
Watkins, C.J.C.H.: Learning from delayed rewards, PhD Thesis, University of Cambridge, England (1989)
Wiering, M.A., Van Hasselt, H.: The QV family compared to other reinforcement learning algorithms. In: IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning, pp. 101–108 (2009)
Young, S., Gašić, M., Keizer, S., Mairesse, F., Schatzmann, J., Thomson, B., Yu, K.: The Hidden Information State model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language 24(2), 150–174 (2010)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2012 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Papangelis, A., Kouroupas, N., Karkaletsis, V., Makedon, F. (2012). An Adaptive Dialogue System with Online Dialogue Policy Learning. In: Maglogiannis, I., Plagianakos, V., Vlahavas, I. (eds) Artificial Intelligence: Theories and Applications. SETN 2012. Lecture Notes in Computer Science(), vol 7297. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-30448-4_41
Download citation
DOI: https://doi.org/10.1007/978-3-642-30448-4_41
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-30447-7
Online ISBN: 978-3-642-30448-4
eBook Packages: Computer ScienceComputer Science (R0)