Book contents
- Frontmatter
- Contents
- List of contributors
- 1 Introduction
- Part I Joint construction
- Part II Reference
- Part III Handling uncertainty
- 7 Reinforcement learning approaches to natural language generation in interactive systems
- 8 A joint learning approach for situated language generation
- Part IV Engagement
- Part V Evaluation and shared tasks
- Author index
- Subject index
- References
8 - A joint learning approach for situated language generation
from Part III - Handling uncertainty
Published online by Cambridge University Press: 05 July 2014
Book contents
- Frontmatter
- Contents
- List of contributors
- 1 Introduction
- Part I Joint construction
- Part II Reference
- Part III Handling uncertainty
- 7 Reinforcement learning approaches to natural language generation in interactive systems
- 8 A joint learning approach for situated language generation
- Part IV Engagement
- Part V Evaluation and shared tasks
- Author index
- Subject index
- References
Summary
Introduction
Interactive systems are increasingly situated: they have knowledge about the non-linguistic context of the interaction, including aspects related to location, time, and the user (Byron and Fosler-Lussier, 2006; Kelleher et al., 2006; Stoia et al., 2006; Raux and Nakano, 2010; Garoufi and Koller, 2011; Janarthanam et al., 2012). This extra knowledge makes it possible for the Natural Language Generation (NLG) components of these systems to be more adaptive, changing their output to suit the larger context. Adaptive NLG systems for situated interaction aim to produce the most effective utterance for each user in each physical and discourse context. At each stage of the generation process (what to say or content selection, how to structure content or utterance planning, and how to express content or surface realization), the best choices depend on the physical and linguistic context, which is constantly changing. Consequently, it is key to successful interaction that adaptive NLG systems constantly monitor the physical environment, the dialogue history, and the user's preferences and behaviors. As the representations of each of these will necessarily be incomplete and error-prone, adaptive NLG systems must also be able to model uncertainty in the generation process.
A designer of adaptive NLG systems faces at least two challenges. The first challenge is to identify the set of contextual features that are relevant to decision making in a specific generation situation. The second challenge is to develop a method for selecting a (near-)optimal choice in any given situation from a set of competing ones that may initially appear as viable alternatives. Complicating these challenges is the fact that individual generation decisions are tightly interrelated, so the best decision at one stage may easily depend on others.
- Type
- Chapter
- Information
- Natural Language Generation in Interactive Systems , pp. 180 - 204Publisher: Cambridge University PressPrint publication year: 2014
References
, , and (2010). A simple domain-independent probabilistic approach to generation. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 502-512, Boston, MA. Association for Computational Linguistics.Google Scholar
, , , and (1970). A maximization technique occurring in the statistical analysis of probabilistic functions of Markov chains. The Annals of Mathematical Statistics, 41(1):164-171.CrossRefGoogle Scholar
and (2006). Comparing automatic and human evaluation of NLG systems. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics (EACL), pages 313-320, Trento, Italy. Association for Computational Linguistics.Google Scholar
and (2006). The OSU Quake 2004 corpus of two-party situated problem-solving dialogs. Technical Report OSU-CISRC-805-TR57, Ohio State University.Google Scholar
(2009). Hierarchical Reinforcement Learning for Spoken Dialogue Systems. PhD thesis, University of Edinburgh.Google Scholar
and (2011a). Optimizing situated dialogue management in unknown environments. In Proceedings of the International Conference on Spoken Language Processing (INTERSPEECH), pages 1009-1012, Florence, Italy. International Speech Communication Association.Google Scholar
and (2011b). Spatially-aware dialogue control using hierarchical reinforcement learning. ACM Transactions on Speech and Language Processing, 7(3):5:1–5:26.CrossRefGoogle Scholar
, , and (2012). Hierarchical dialogue policy learning using flexible state transitions and linear function approximation. In Proceedings of the International Conference on Computational Linguistics (COLING), pages 95-102, Mumbai, India. International Committee on Computational Linguistics.Google Scholar
, , , and (2005). Human-computer dialogue simulation using hidden Markov models. In Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pages 290-295, San Juan, Puerto Rico. Institute of Electrical and Electronics Engineers.Google Scholar
, , and (2004). Fast reinforcement learning of dialogue policies using stable function approximation. In Proceedings of the International Joint Conference on Natural Language Processing (IJCNLP), pages 1-11, Hainan Island, China. Association for Computational Linguistics.Google Scholar
(2010). Generating referring expressions with reference domain theory. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 27-36, Trim, Ireland. Association for Computational Linguistics.Google Scholar
and (2010). Hierarchical reinforcement learning for adaptive text generation. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 37-46, Trim, Ireland. Association for Computational Linguistics.Google Scholar
and (2011a). Combining hierarchical reinforcement learning and Bayesian networks for natural language generation in situated dialogue. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 110-120, Nancy, France. Association for Computational Linguistics.Google Scholar
and (2011b). Hierarchical reinforcement learning and hidden Markov models for task-oriented natural language generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT), pages 654-659, Portland, OR. Association for Computational Linguistics.Google Scholar
and (2012). Comparing HMMs and Bayesian Networks for surface realisation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL-HLT), pages 636-640, Montréal, Canada. Association for Computational Linguistics.Google Scholar
, , and (2011). Optimising natural language generation decision making for situated dialogue. In Proceedings of the SIGdial Conference on Discourse and Dialogue (SIGDIAL), pages 78-87, Portland, OR. Association for Computational Linguistics.Google Scholar
, , and (2008). Practical grammar-based NLG from examples. In Proceedings of the International Workshop on Natural Language Generation (INLG), pages 77-85, Salt Fork, OH. Association for Computational Linguistics.Google Scholar
(2000). Hierarchical reinforcement learning with the MAXQ value function decomposition. Journal of Artificial Intelligence Research, 13:227-303.Google Scholar
, , , and (2010). The GIVE-2 corpus of giving instructions in virtual environments. In Proceedings of the International Conference on Language Resources and Evaluation (LREC), Valletta, Malta. European Language Resources Association.Google Scholar
and (2010). Automated planning for situated natural language generation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1573-1582, Uppsala, Sweden. Association for Computational Linguistics.Google Scholar
and (2011). Combining symbolic and corpus-based approaches for the generation of successful referring expressions. In Proceedings of the European Workshop on Natural Language Generation (ENLG), pages 121-131. Nancy, France. Association for Computational Linguistics.Google Scholar
, , , , and (2011). On-line policy optimisation of spoken dialogue systems via live interaction with human subjects. In Proceedings of the IEEE workshop on Automatic Speech Recognition and Understanding (ASRU), pages 312-317, Waikoloa, HI. Institute of Electrical and Electronics Engineers.Google Scholar
, , and (2005). Hybrid reinforcement/supervised learning for dialogue policies from Communicator data. In Proceedings of the Workshop on Knowledge and Reasoning in Practical Dialogue Systems (KRPDS), pages 68-75, Edinburgh, Scotland. International Joint Conference on Artificial Intelligence.Google Scholar
and (2010). Learning adaptive referring expression generation policies for spoken dialogue systems. In and , editors, Empirical Methods in Natural Language Generation, pages 67-84. Springer, Berlin.Google Scholar
, , and (2012). A web-based evaluation framework for spatial instruction-giving systems. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 49-54, Jeju Island, Korea. Association for Computational Linguistics.Google Scholar
, , and (2006). Incremental generation of spatial referring expressions in situated dialog. In Proceedings of the International Conference on Computational Linguistics and the Annual Meeting of the Association for Computational Linguistics (COLING-ACL), pages 745-752, Sydney, Australia. Association for Computational Linguistics.Google Scholar
, , , , , and (2010a). The first challenge on generating instructions in virtual environments. In and , editors, Empirical Methods in Natural Language Generation, pages 328-352. Springer LNCS, Berlin, Germany.Google Scholar
, , , , , , , and (2010b). Report on the second NLG challenge on generating instructions in virtual environments (GIVE-2). In Proceedings of the International Conference on Natural Language Generation (INLG), pages 243-250, Trim, Ireland. Association for Computational Linguistics.Google Scholar
(2008). Adaptive natural language generation in dialogue using reinforcement learning. In Proceedings of the Workshop on the Semantics and Pragmatics of Dialogue (SEMDIAL), pages 141-148, London, UK. SemDial.Google Scholar
(2011). Learning what to say and how to say it: Joint optimisation of spoken dialogue management and natural language generation. Computer Speech & Language, 25(2):210-221.CrossRefGoogle Scholar
, , and (2000). A stochastic model of human-machine interaction for learning dialog strategies. IEEE Transactions on Speech and Audio Processing, 8(1):11-23.CrossRefGoogle Scholar
, , , , , , and (2010). Phrase-based statistical language generation using graphical models and active learning. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), pages 1552-1561, Uppsala, Sweden. Association for Computational Linguistics.Google Scholar
, , , and (2011). Sample-efficient batch reinforcement learning for dialogue management optimization. ACM Transactions on Speech and Language Processing (Special Issue on Machine Learning for Robust and Adaptive Spoken Dialogue Systems), 7(3):7.Google Scholar
(1989). Tutorial on hidden Markov models and selected applications in speech recognition. Proceedings of the IEEE, 77(2):257-286.CrossRefGoogle Scholar
and (2010). The dynamics of action corrections in situated interaction. In Proceedings of the SIGdial Conference on Discourse and Dialogue (SIGDIAL), pages 165-174, Tokyo, Japan. Association for Computational Linguistics.Google Scholar
(1998). A relevance theoretic approach to reference. In Proceedings of the Relevance Theory Workshop, pages 45-50, Luton, UK. University of Luton.Google Scholar
and (2011). Learning and evaluation of dialogue strategies for new applications: Empirical methods for optimization from small data sets. Computational Linguistics, 37(1):153-196.CrossRefGoogle Scholar
and (2001). Reference resolution within the framework of cognitive grammar. In Proceedings of the International Colloquium on Cognitive Science, Donostia -San Sebastian, Spain. Institute for Logic, Cognition, Language and Information (ILCLI) and the Dept. of Logic and Philosophy of Science of the University of the Basque Country.Google Scholar
and (2007). An NLG evaluation competition? Eight reasons to be cautious. In Proceedings of the NSF Workshop onShared Tasks and Comparative EvaluationinNatural Language Generation, Arlington, VA. National Science Foundation.Google Scholar
, , and (2008). Where do the words come from? Learning models for word choice and ordering from spoken dialog corpora. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), pages 5037-5040, Las Vegas, NV. Institute of Electrical and Electronics Engineers.Google Scholar
, , , and (2006). Noun phrase generation for situated dialogs. In Proceedings of the International Conference on Natural Language Generation (INLG), pages 81-88, Sydney, Australia. Association for Computational Linguistics.Google Scholar
(1996). Generalization in reinforcement learning: Successful examples using sparse coarse coding. In , , and , editors, Advances in Neural Information Processing Systems, pages 1038-1044. MIT Press, Cambridge, MA.Google Scholar
(2009). Statistical Methods for Spoken Dialogue Management. PhD thesis, University of Cambridge.Google Scholar
(2000). Bootstrapping syntax and recursion using alignment-based learning. In Proceedings of the International Conference on Machine Learning (ICML), pages 1063-1070, Stanford, CA. The International Machine Learning Society.Google Scholar
(1989). Learning from Delayed Rewards. PhD thesis, Kings College, University of Cambridge.Google Scholar
(2006). Partially Observable Markov Decision Processes for Spoken Dialogue Management. PhD thesis, University of Cambridge.Google Scholar
, , , , , , and (2010). The hidden information state model: A practical framework for POMDP-based spoken dialogue management. Computer Speech & Language, 24(2):150-174.CrossRefGoogle Scholar
- 1
- Cited by