Abstract
One of the main problems with partially observable Markov decision process (POMDP) in development of spoken dialog system (SDS) is lack of scalability. In development of an SDS with electronic program guide (EPG) domain, we devised a POMDP approach which is operated with summary spaces to respond accurately to multiple drifting goals and massive numbers of slot entities. The main point of the proposed approach is to introduce a hybrid architecture that is implemented by a meta-action selector and a service provider. A trained POMDP policy was used to select meta-actions. The selected meta-actions were transformed to the system action in the service provider, which is implemented with the given system action model. By using this architecture, various system actions could be elicited with reduced complexity in the dialog process. We trained the system with the specified simulator and observed its behavior with learning curves in the Korean EPG domain. The convergence of learning curve implies the feasibility of our approach in commercial EPG domain SDS.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The notation is suggested by the previous research (Thomson and Young 2010).
References
Bellman R (1957) A Markovian decision process. Indiana Univ Math J 6:679–684
Bui T, Zwiers J, Poel M, Nijholt A (2006) Toward affective dialogue modeling using partially observable Markov decision processes. In: Reichardt D, Levi P, Meyer JJ (eds) Proceedings of the 1st workshop on emotion and computing? Current research and future impact. University of Bremen, Bremen, pp 47–50. http://www.doc.utwente.nl/66792/
Bui T, Poel M, Nijholt A, Zwiers J (2009) A tractable hybrid ddn-pomdp approach to affective dialogue modeling for probabilistic frame-based dialogue systems. Nat Lang Eng 15(2):273–307
Gasic M, Breslin C, Henderson M, Kim D, Szummer M, Thomson B, Tsiakoulis P, Young S (2013) Pomdp-based dialogue manager adaptation to extended domains. In: Proceedings of the SIGDIAL 2013 conference. Association for Computational Linguistics, Metz, pp 214–222. http://www.aclweb.org/anthology/W/W13/W13-4035
Peters J, Schaal S (2008) Natural actor-critic. Neurocomputing 71(7–9):1180–1190. doi:http://dx.doi.org/10.1016/j.neucom.2007.11.026. http://www.sciencedirect.com/science/article/pii/S0925231208000532. Progress in modeling, theory, and application of computational intelligence. 15th European symposium on artificial neural networks 2007. 15th European symposium on artificial neural networks 2007
Roy N, Pineau J, Thrun S (2000) Spoken dialogue management using probabilistic reasoning. In: Proceedings of the 38th annual meeting on association for computational linguistics, ACL ’00. Association for Computational Linguistics, Stroudsburg, PA, pp 93–100. doi:10.3115/1075218.1075231. http://www.dx.doi.org/10.3115/1075218.1075231
Schatzmann J, Thomson B, Weilhammer K, Ye H, Young S (2007) Agenda-based user simulation for bootstrapping a pomdp dialogue system. In: Human language technologies 2007: the conference of the North American chapter of the association for computational linguistics; companion volume, short papers, NAACL-Short ’07. Association for Computational Linguistics, Stroudsburg, PA, pp 149–152
Thomson B, Young S (2010) Bayesian update of dialogue state: a pomdp framework for spoken dialogue systems. Comput Speech Lang 24(4):562–588. doi:10.1016/j.csl.2009.07.003. http://www.dx.doi.org/10.1016/j.csl.2009.07.003
Wang Z, Lemon O (2013) A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information. In: Proceedings of the SIGDIAL 2013 conference. Association for Computational Linguistics, Metz, pp 423–432. http://www.aclweb.org/anthology/W13-4067
Williams JD, Young S (2007) Partially observable Markov decision processes for spoken dialog systems. Comput Speech Lang 21(2):393–422
Young S (2006) Using pomdps for dialog management. In: Spoken language technology workshop, 2006. IEEE, pp 8–13. doi:10.1109/SLT.2006.326785
Young S, Schatzmann J, Weilhammer K, Ye H (2007) The hidden information state approach to dialog management. In: IEEE international conference on acoustics, speech and signal processing, 2007. ICASSP 2007, vol 4, pp. IV–149–IV–152. doi:10.1109/ICASSP.2007.367185
Young S, Gašić M, Keizer S, Mairesse F, Schatzmann J, Thomson B, Yu K (2010) The hidden information state model: a practical framework for pomdp-based spoken dialogue management. Comput Speech Lang 24(2):150–174. doi:10.1016/j.csl.2009.04.001. http://www.dx.doi.org/10.1016/j.csl.2009.04.001
Zhang B, Cai Q, Mao J, Guo B (2001) Planning and acting under uncertainty: a new model for spoken dialogue systems. In: Proceedings of the 17th conference on uncertainty in artificial intelligence, UAI’01. Morgan Kaufmann, San Francisco, CA, pp 572–579. http://www.dl.acm.org/citation.cfm?id=2074022.2074092
Acknowledgements
This work was supported by National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIP) (NRF-2014R1A2A1A01003041).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Koo, S., Ryu, S., Lee, K., Lee, G.G. (2015). Scalable Summary-State POMDP Hybrid Dialog System for Multiple Goal Drifting Requests and Massive Slot Entity Instances. In: Lee, G., Kim, H., Jeong, M., Kim, JH. (eds) Natural Language Dialog Systems and Intelligent Assistants. Springer, Cham. https://doi.org/10.1007/978-3-319-19291-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-19291-8_8
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19290-1
Online ISBN: 978-3-319-19291-8
eBook Packages: Computer ScienceComputer Science (R0)