Yanxiang Ling
Skip Abstract Section
Abstract
Recent research has highlighted the importance of mixed-initiative interactions in conversational search. To enable mixed-initiative interactions, information retrieval systems should be able to ask diverse questions, such as information-seeking, clarification, and open-ended ones. question generation (QG) of open-domain conversational systems aims at enhancing the interactiveness and persistence of human-machine interactions. The task is challenging because of the sparsity of question generation (QG)-specific data in conversations. Current work is limited to single-turn interaction scenarios. We propose a context-enhanced neural question generation(CNQG) model that leverages the conversational context to predict question content and pattern, then perform question decoding. A hierarchical encoder framework is employed to obtain the discourse-level context representation. Based on this, we propose Review and Transit mechanisms to respectively select contextual keywords and predict new topic words to further construct the question content. Conversational context and the predicted question content are used to produce the question pattern, which in turn guides the question decoding process implemented by a recurrent decoder with a joint attention mechanism. To fully utilize the limited QG-specific data to train our question generator, we perform multi-task learning with three auxiliary training objectives, i.e., question pattern prediction, Review, and Transit mechanisms. The required additional labeled data is obtained in a self-supervised way. We also design a weight decaying strategy to adjust the influences of various auxiliary learning tasks. To the best of our acknowledge, we are the first to extend the application of QG to the multi-turn open-domain conversational scenario. Extensive experimental results demonstrate the effectiveness of our proposal and its main components on generating relevant and informative questions, with robust performance for contexts with various lengths.
- [1] . 2020. ConvAI3: Generating clarifying questions for open-domain dialogue systems (ClariQ). CoRR abs/2009.11352 (2020).Google Scholar
- [2] . 2019. Asking clarifying questions in open-domain information-seeking conversations. In Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. Association for Computing Machinery, 475–484.Google ScholarDigital Library
- [3] . 2018. Conceptualizing agent-human interactions during the conversational search process. In The 2nd Workshop on Conversational Approaches to Information Retrieval.Google Scholar
- [4] . 1980. Anomalous states of knowledge as a basis for information retrieval. Can. J. Inf. Sci. 5, 1 (1980), 133–143.Google Scholar
- [5] . 2019. Conversational product search based on negative feedback. In Proceedings of the 28th ACM International Conference on Information & Knowledge Management (CIKM’19). 359–368.Google ScholarDigital Library
- [6] . 1997. Multitask learning. Mach. Learn. 28, 1 (1997), 41–75. Google ScholarDigital Library
- [7] . 2017. A survey on dialogue systems: Recent advances and new frontiers. SIGKDD Explor. 19, 2 (2017), 25–35.Google ScholarDigital Library
- [8] . 2018. Hierarchical variational memory network for dialogue generation. In Proceedings of the 2018 World Wide Web Conference. 1653–1662.Google ScholarDigital Library
- [9] . 2014. On the properties of neural machine translation: Encoder–decoder approaches. In Proceedings of the 8th Workshop on Syntax, Semantics and Structure in Statistical Translation. 103–111.Google ScholarCross Ref
- [10] . 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. 1724–1734.Google ScholarCross Ref
- [11] . 2016. Towards conversational recommender systems. In KDD 2016: 22nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining. ACM, 815–824.Google ScholarDigital Library
- [12] . 1989. Word association norms, mutual information and lexicography. In 27th Annual Meeting of the Association for Computational Linguistic. 76–83.Google Scholar
- [13] . 1987. I3R: A new approach to the design of document retrieval systems. J. Assoc. Inf. Sci. Technol. 38, 6 (1987), 389–404.Google ScholarDigital Library
- [14] . 2020. Syn-QG: Syntactic and shallow semantic rules for question generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 752–765.Google ScholarCross Ref
- [15] . 2017. Learning to ask: Neural question generation for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 1342–1352.Google ScholarCross Ref
- [16] . 2017. Question generation for question answering. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. 866–874.Google ScholarCross Ref
- [17] . 2021. Advances and challenges in conversational recommender systems: A survey. AI Open 2 (
July 2021), 100–126.Google ScholarCross Ref - [18] . 2019. Interconnected question generation with coreference alignment and conversation flow modeling. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 4853–4862.Google ScholarCross Ref
- [19] . 2018. Aspect-based question generation. In 6th International Conference on Learning Representations.Google Scholar
- [20] . 2020. Challenges in building intelligent open-domain dialog systems. ACM Trans. Inf. Syst. (TOIS) 38, 3 (2020), 1–32.Google ScholarDigital Library
- [21] . 2018. Multi-task learning using uncertainty to weigh losses for scene geometry and semantics. In 2018 IEEE Conference on Computer Vision and Pattern Recognition. 7482–7491.Google Scholar
- [22] . 2018. Toward voice query clarification. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. Association for Computing Machinery, 1257–1260.Google ScholarDigital Library
- [23] . 2020. Analysing the effect of clarifying questions on document ranking in conversational search. In Proceedings of the 2020 ACM SIGIR on International Conference on Theory of Information Retrieval. 129–132.Google ScholarDigital Library
- [24] . 2020. PONE: A novel automatic evaluation metric for open-domain generative dialogue systems. CoRR abs/2004.02399 (2020).Google Scholar
- [25] . 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining (WSDM’20). 304–312.Google ScholarDigital Library
- [26] . 2016. A diversity-promoting objective function for neural conversation models. In The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 110–119.Google ScholarCross Ref
- [27] . 2017. Learning through dialogue interactions by asking questions. In 5th International Conference on Learning Representations.Google Scholar
- [28] . 2017. DailyDialog: A manually labelled multi-turn dialogue dataset. In Proceedings of the E8th International Joint Conference on Natural Language Processing. 986–995.Google Scholar
- [29] . 2003. Automatic evaluation of summaries using n-gram co-occurrence statistics. In Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics (HLT-NAACL’03).Google ScholarDigital Library
- [30] . 2017. Focal loss for dense object detection. In Proceedings of the IEEE International Conference on Computer Vision. 2980–2988.Google ScholarCross Ref
- [31] . 2020. Leveraging context for neural question generation in open-domain dialogue systems. In WWW’20: The Web Conference 2020. 2486–2492.Google ScholarDigital Library
- [32] . 2021. Context-controlled topic-aware neural response generation for open-domain dialog systems. Inf. Process. & Manage. 58, 1 (2021), 102392.Google ScholarCross Ref
- [33] . 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. 2122–2132.Google ScholarCross Ref
- [34] . 2019. Multi-task deep neural networks for natural language understanding. In Proceedings of the 57th Conference of the Association for Computational Linguistics, , , and (Eds.). 4487–4496.Google ScholarCross Ref
- [35] . 2020. Transformer-based end-to-end question generation. CoRR abs/2005.01107 (2020).Google Scholar
- [36] . 2018. A neural local coherence model for text quality assessment. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 4328–4339.Google ScholarCross Ref
- [37] . 2019. Towards answer-unaware conversational question generation. In Proceedings of the 2nd Workshop on Machine Reading for Question Answering (MRQA@EMNLP’19). 63–71.Google ScholarCross Ref
- [38] . 2019. Reinforced dynamic reasoning for conversational question generation. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 2114–2124.Google ScholarCross Ref
- [39] . 2020. Semantic graphs for generating deep questions. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 1463–1475.Google ScholarCross Ref
- [40] . 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. 311–318.Google ScholarDigital Library
- [41] . 2017. A theoretical framework for conversational search. In Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval (CHIIR’17). 117–126.Google ScholarDigital Library
- [42] . 2019. Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR abs/1910.10683 (2019).Google Scholar
- [43] . 2019. Towards empathetic open-domain conversation models: A new benchmark and dataset. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 5370–5381.Google ScholarCross Ref
- [44] . 2021. Conversations with search engines. ACM Trans. Inf. Syst. 30, 2 (2021).Google Scholar
- [45] . 2018. Preference elicitation as an optimization problem. In RecSys 2018: The ACM Conference on Recommender Systems. ACM, 172–180.Google ScholarDigital Library
- [46] . 2016. Generating factoid questions with recurrent neural networks: The 30M factoid question-answer corpus. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Google ScholarCross Ref
- [47] . 2018. A survey of available corpora for building data-driven dialogue systems: The journal version. Dialogue Discourse 9, 1 (2018), 1–49.Google ScholarCross Ref
- [48] . 2016. Building end-to-end dialogue systems using generative hierarchical neural network models. In Proceedings of the 30th AAAI Conference on Artificial Intelligence. 3776–3783.Google ScholarDigital Library
- [49] . 2017. A hierarchical latent variable encoder-decoder model for generating dialogues. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 3295–3301.Google ScholarCross Ref
- [50] . 2015. Neural responding machine for short-text conversation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing. 1577–1586.Google ScholarCross Ref
- [51] . 2015. A neural network approach to context-sensitive generation of conversational responses. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 196–205.Google ScholarCross Ref
- [52] . 2018. Answer-focused and position-aware neural question generation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. 3930–3939.Google ScholarCross Ref
- [53] . 2014. Sequence to sequence learning with neural networks. In Proceedings of the 27th International Conference on Neural Information Processing Systems. 3104–3112.Google ScholarDigital Library
- [54] . 2018. Learning to collaborate for question answering and asking. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, , , and (Eds.). 1564–1574.Google ScholarCross Ref
- [55] . 2018. Style and alignment in information-seeking conversation. In Proceedings of the 2018 Conference on Human Information Interaction and Retrieval (CHIIR’18). 42—51.Google ScholarDigital Library
- [56] . 2018. Informing the design of spoken conversational search: Perspective paper. In Proceedings of the 2018 Conference on Human Information Interaction & Retrieval (CHIIR’18). 32–41.Google ScholarDigital Library
- [57] . 2021. A large-scale analysis of mixed initiative in information-seeking dialogues for conversational search. ACM Trans. Inf. Syst. 39, 4 (
August 2021), Article 49.Google ScholarDigital Library - [58] . 2017. Conversational exploratory search via interactive storytelling. In 1st International Workshop on Search-Oriented Conversational AI.Google Scholar
- [59] . 2019. QRFA: A data-driven model of information-seeking dialogues. In ECIR 2019: 41st European Conference on Information Retrieval. Springer, 541–557.Google ScholarDigital Library
- [60] . 2020. Conversational browsing. arXiv:2012.03704. https://arxiv.org/abs/2012.03704.Google Scholar
- [61] . 2019. Answer-guided and semantic coherent question generation in open-domain conversation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 5065–5075.Google ScholarCross Ref
- [62] . 2018. Chat more: Deepening and widening the chatting topic via a deep model. In The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval. 255–264.Google ScholarDigital Library
- [63] . 2018. Learning to ask questions in open-domain conversational systems with typed decoders. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2193–2203.Google ScholarCross Ref
- [64] . 2021. Kappa coefficients for dichotomous-nominal classifications. Adv. Data Anal. Classif. 15, 1 (2021), 193–208.Google ScholarDigital Library
- [65] . 2008. Interpreting tf-idf term weights as making relevance decisions. ACM Trans. Inf. Syst. (TOIS) 26, 3 (2008), 1–37.Google ScholarDigital Library
- [66] . 2017. Topic aware neural response generation. In Proceedings of the 31st AAAI Conference on Artificial Intelligence. 3351–3357.Google ScholarCross Ref
- [67] . 2018. Hierarchical recurrent attention network for response generation. In Proceedings of the 32nd AAAI Conference on Artificial Intelligence. 5610–5617.Google ScholarCross Ref
- [68] . 2019. Asking clarification questions in knowledge-based question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP’19). 1618–1629.Google ScholarCross Ref
- [69] . 2019. A cross-domain transferable neural coherence model. In Proceedings of the 57th Conference of the Association for Computational Linguistics. 678–687.Google ScholarCross Ref
- [70] . 2020. Generating clarifying questions for information retrieval. In Proceedings of the 29th International Conference on World Wide Web
(WWW’20) .Google ScholarDigital Library - [71] . 2019. ReCoSa: Detecting the relevant contexts with self-attention for multi-turn dialogue generation. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 3721–3730.Google ScholarCross Ref
- [72] . 2020. Grounded conversation generation as guided traverses in commonsense knowledge graphs. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2031–2043.Google ScholarCross Ref
- [73] . 2018. Personalizing dialogue agents: I have a dog, do you have pets too?. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. 2204–2213.Google ScholarCross Ref
- [74] . 2020. BERTScore: Evaluating text generation with BERT. In 8th International Conference on Learning Representations (ICLR’20).Google Scholar
- [75] . 2018. Towards conversational search and recommendation: System ask, user respond. In Proceedings of the 27th ACM International Conference on Information & Knowledge Management (CIKM’18). 177–186.Google ScholarDigital Library
- [76] . 2020. DIALOGPT: Large-scale generative pre-training for conversational response generation. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations. 270–278.Google ScholarCross Ref
- [77] . 2017. Learning discourse-level diversity for neural dialog models using conditional variational autoencoders. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics. 654–664.Google ScholarCross Ref
- [78] . 2017. Neural question generation from text: A preliminary study. In Natural Language Processing and Chinese Computing—6th CCF International Conference. 662–671.Google Scholar
- [79] . 2019. Multi-task learning with language modeling for question generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, , , , and (Eds.). 3392–3397.Google ScholarCross Ref
- [80] . 2019. Question-type driven question generation. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. 6031–6036.Google ScholarCross Ref
Index Terms
- Generating Relevant and Informative Questions for Open-Domain Conversations
Recommendations
Analyzing and Characterizing User Intent in Information-seeking Conversations
SIGIR '18: The 41st International ACM SIGIR Conference on Research & Development in Information RetrievalUnderstanding and characterizing how people interact in information-seeking conversations is crucial in developing conversational search systems. In this paper, we introduce a new dataset designed for this purpose and use it to analyze information-...
Harnessing Evolution of Multi-Turn Conversations for Effective Answer Retrieval
CHIIR '20: Proceedings of the 2020 Conference on Human Information Interaction and RetrievalWith the improvements in speech recognition and voice generation technologies over the last years, a lot of companies have sought to develop conversation understanding systems that run on mobile phones or smart home devices through natural language ...
Asking Clarifying Questions in Open-Domain Information-Seeking Conversations
SIGIR'19: Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information RetrievalUsers often fail to formulate their complex information needs in a single query. As a consequence, they may need to scan multiple result pages or reformulate their queries, which may be a frustrating experience. Alternatively, systems can improve user ...
Comments