skip to main content
10.1145/3589335.3651905acmconferencesArticle/Chapter ViewAbstractPublication PagesthewebconfConference Proceedingsconference-collections
research-article

Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems

Published: 13 May 2024 Publication History

Abstract

When interacting with Retrieval-Augmented Generation (RAG)-based conversational agents, the users must carefully craft their queries to be understood correctly. Yet, understanding the system's capabilities can be challenging for the users, leading to ambiguous questions that necessitate further clarification. This work aims to bridge the gap by developing a suggestion question generator. To generate suggestion questions, our approach involves utilizing dynamic context, which includes both dynamic few-shot examples and dynamically retrieved contexts. Through experiments, we show that the dynamic contexts approach can generate better suggestion questions as compared to other prompting approaches.

Supplemental Material

MP4 File
Presentation video

References

[1]
Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar,Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. QuAC: question answering in context. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Brussels, Belgium.
[2]
Siva Reddy, Danqi Chen, and Christopher D. Manning. 2019. CoQA: a conversational question answering challenge. Transactions of the Association for Computational Linguistics, 7, 249--266.
[3]
Yue Feng, Hossein A Rahmani, Aldo Lipani, and Emine Yilmaz. 2023. Towards asking clarification questions for information seeking on task-oriented dialogues. arXiv preprint arXiv:2305.13690.
[4]
Tom Brown et al. 2020. Language models are few-shot learners. In Advances in Neural Information Processing Systems. Vol. 33. Curran Associates, Inc., 1877--1901.
[5]
Hongwei Zeng, Bifan Wei, Jun Liu, and Weiping Fu. 2023. Synthesize, prompt and transfer: zero-shot conversational question generation with pre-trained language model. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Toronto, Canada, (July 2023), 8989--9010. /2023.acl-long.500.
[6]
Jingjing Xu, Yuechen Wang, Duyu Tang, Nan Duan, Pengcheng Yang, Qi Zeng, Ming Zhou, and Xu Sun. 2019. Asking clarification questions in knowledge based question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP). Association for Computational Linguistics, Hong Kong, China, (Nov. 2019), 1618--1629.
[7]
Sudha Rao and Hal Daumé III. 2018. Learning to ask good questions: ranking clarification questions using neural expected value of perfect information. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Melbourne, Australia, (July 2018), 2737--2746.
[8]
Sewon Min, Julian Michael, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2020. AmbigQA: answering ambiguous open-domain questions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Online, (Nov. 2020), 5783--5797.
[9]
Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2022. CLAM: selective clarification for ambiguous questions with large language models. CoRR, abs/2212.07769. arXiv: 2212.07769.
[10]
Manas Gaur, Kalpa Gunaratna, Vijay Srinivasan, and Hongxia Jin. 2022. ISEEQ: information seeking question generation using dynamic meta-information retrieval and knowledge graphs. In Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, February 22 - March 1, 2022. AAAI Press, 10672--10680.
[11]
Yosi Mass, Doron Cohen, Asaf Yehudai, and David Konopnicki. 2022. Conversational search with mixed-initiative - asking good clarification questions backed-up by passage retrieval. In Proceedings of the Second DialDoc Workshop on Document-grounded Dialogue and Conversational Question Answering. Association for Computational Linguistics, Dublin, Ireland, (May 2022), 65--71.
[12]
Hossein A. Rahmani, Xi Wang, Yue Feng, Qiang Zhang, Emine Yilmaz, and Aldo Lipani. 2023. A survey on asking clarification questions datasets in conversational systems. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Toronto, Canada, (July 2023), 2698--2716. /2023.acl-long.152.
[13]
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense passage retrieval for opendomain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). Online, 6769--6781.
[14]
Yang Deng, Wenqiang Lei, Lizi Liao, and Tat-Seng Chua. 2023. Prompting and evaluating large language models for proactive dialogues: clarification, target-guided, and non-collaboration. (2023). arXiv: 2305.13626 [cs.CL].
[15]
Mor Geva, Ankit Gupta, and Jonathan Berant. 2020. Injecting numerical reasoning skills into language models. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Online, 946--958.

Cited By

View all
  • (2024)[PromptEng] First International Workshop on Prompt Engineering for Pre-Trained Language ModelsCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3641292(1311-1312)Online publication date: 13-May-2024
  • (2024)A Comparative Analysis of Large Language Models with Retrieval-Augmented Generation based Question Answering System2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)10.1109/I-SMAC61858.2024.10714814(792-798)Online publication date: 3-Oct-2024

Index Terms

  1. Dynamic Contexts for Generating Suggestion Questions in RAG Based Conversational Systems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    WWW '24: Companion Proceedings of the ACM Web Conference 2024
    May 2024
    1928 pages
    ISBN:9798400701726
    DOI:10.1145/3589335
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 13 May 2024

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. conversational systems
    2. few-shot
    3. prompting
    4. question generation
    5. rag

    Qualifiers

    • Research-article

    Conference

    WWW '24
    Sponsor:
    WWW '24: The ACM Web Conference 2024
    May 13 - 17, 2024
    Singapore, Singapore

    Acceptance Rates

    Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)331
    • Downloads (Last 6 weeks)26
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)[PromptEng] First International Workshop on Prompt Engineering for Pre-Trained Language ModelsCompanion Proceedings of the ACM Web Conference 202410.1145/3589335.3641292(1311-1312)Online publication date: 13-May-2024
    • (2024)A Comparative Analysis of Large Language Models with Retrieval-Augmented Generation based Question Answering System2024 8th International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud) (I-SMAC)10.1109/I-SMAC61858.2024.10714814(792-798)Online publication date: 3-Oct-2024

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media