Abstract
This paper reports on an implementation of methods for generating indirect responses in question-answering dialogue based on domain-level strategic reasoning. User’s questions are interpreted as reflexes of underlying user requirements which are potentially satisfied by information beyond what is directly asked about. We find that the algorithms that reason about user requirements yield significantly shorter dialogues than a simpler baseline, and that users are able to interact with these systems in a pragmatically natural way.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
(Stevens et al. 2014) use “\(d\in D\)” to denote requirement sets, since in their framework these sets are shorthand representations of decision problems, which are formally more elaborate and do not need to be elucidated here. While we refer to these sets simply as requirements, we carry their notation over here.
- 2.
- 3.
- 4.
These subjects simply asked all possible questions for most flats, failing to reject even if it was immediately clear that the flat under discussion was sub-optimal.
- 5.
Early feedback on the system indicated that the indirect answers were more natural when preceded by a discourse connective like “well”, which we included for our evaluations. Shortly before the time of writing we ran the same experiment again but without the discourse connective. Looking only at those subjects who finished the task (50 total), the effects reported here were replicated; however, in contrast with the first experiment, when we looked at all subjects who asked more than a single question (72 total), we found a large group effect (a 69 % decrease for the pragmatic group) on whether the subjects successfully completed the task (a mixed effects binomial regression yields significance, with \(z=-2.19,\,p=0.03\)). This is consistent with initial intuitions that the discourse marker makes the answers more natural, thus making the task easier and/or more pleasant.
References
Allen, J.F., Perrault, C.R.: Analyzing intention in utterances. Artif. Intell. 15(3), 143–178 (1980)
Benz, A., Bertomeu, N., Strekalova, A.: A decision-theoretic approach to finding optimal responses to over-constrained queries in a conceptual search space. In: Proceedings of the 15th Workshop on the Semantics and Pragmatics of Dialogue, pp. 37–46 (2011)
Benz, A., van Rooij, R.: Optimal assertions, and what they implicate: a uniform game theoretic approach. Topoi 26(1), 63–78 (2007)
Bertomeu, N., Uszkoreit, H., Frank, A., Krieger, H.-U., Jörg, B.: Contextual phenomena and thematic relations in database QA dialogues: results from a wizard-of-oz experiment. In: Proceedings of the Interactive Question Answering Workshop at HLT-NAACL, pp. 1–8 (2006)
Chai, J.Y. Jin, R.: Discourse structure for context question answering. In: Proceedings of the Workshop on Pragmatics of Question Answering at HLT-NAACL, pp. 23–30 (2004)
de Marneffe, M.-C., Grimm, S., Potts, C.: Not a simple yes or no: uncertainty in indirect answers. In: Proceedings of the SIGDIAL 2009 Conference: The 10th Annual Meeting of the Special Interest Group on Discourse and Dialogue, pp. 136–143 (2009)
Gaasterland, T., Godfrey, P., Minker, J.: An overview of cooperative answering. J. Intell. Inf. Syst. 1(2), 123–157 (1992)
Green, N., Carberry, S.: Generating indirect answers to yes-no questions. In: Proceedings of the 7th International Workshop on Natural Language Generation, pp. 189–198 (1994)
Hoffart, J., Suchanek, F.M., Berberich, K., Lewis-Kelham, E., De Melo, G., Weikum, G.: YAGO2: exploring and querying world knowledge in time, space, context, and many languages. In: Proceedings of the 20th International Conference Companion on World Wide Web, pp. 229–232. Association for Computational Linguistics (2011)
Larsson, S., Traum, D.R.: Information state and dialogue management in the TRINDI dialogue move engine toolkit. Nat. Lang. Eng. 6(3&4), 323–340 (2000)
Moriceau, V.: Generating intelligent numerical answers in a question-answering system. In: Proceedings of the 4th International Natural Language Generation Conference, pp. 103–110. Association for Computational Linguistics (2006)
Popescu, A.-M., Etzioni, O., Kautz, H.: Towards a theory of natural language interfaces to databases. In: Proceedings of the 8th International Conference on Intelligent User Interfaces, pp. 149–157. ACM (2003)
Potts, C.: Indirect answers and cooperation: on asher and lascarides’s making the right commitments in dialogue. http://www.stanford.edu/cgpotts/commentaries/potts-umich08-cmts-on-asherlascarides.pdf (2008). Accessed 2 Sept 2013
Stevens, J.S., Benz, A., Reusse, S., Laarmann-Quante, R., Klabunde, R.: Indirect answers as potential solutions to decision problems. In: Proceedings of the 18th Workshop on the Semantics and Pragmatics of Dialogue, pp. 145–153 (2014)
Traum, D.R., Larsson, S.: The information state approach to dialogue management. In: van Kuppevelt, J., Smith, R.W. (eds.) Current and New Directions in Discourse and Dialogue. Text, Speech and Language Technology, pp. 325–353. Springer, Netherlands (2003)
van Rooij, R.: Questioning to resolve decision problems. Linguist. Philos. 26(6), 727–763 (2003)
Yao, X., Berant, J., Van Durme, B.: Freebase QA: information extraction or semantic parsing?. In: Proceedings of the ACL 2014 Workshop on Semantic Parsing, pp. 82–86. Association for Computational Linguistics (2014)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Stevens, J.S., Benz, A., Reuße, S., Klabunde, R., Raithel, L. (2015). Pragmatic Query Answering: Results from a Quantitative Evaluation. In: Biemann, C., Handschuh, S., Freitas, A., Meziane, F., Métais, E. (eds) Natural Language Processing and Information Systems. NLDB 2015. Lecture Notes in Computer Science(), vol 9103. Springer, Cham. https://doi.org/10.1007/978-3-319-19581-0_9
Download citation
DOI: https://doi.org/10.1007/978-3-319-19581-0_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19580-3
Online ISBN: 978-3-319-19581-0
eBook Packages: Computer ScienceComputer Science (R0)