skip to main content
research-article

Spoken conversational search: audio-only interactive information retrieval

Published: 23 March 2021 Publication History

Abstract

Speech-based web search where no keyboard or screens are available to present search engine results is becoming ubiquitous, mainly through the use of mobile devices and intelligent assistants such as Apple's HomePod, Google Home, or Amazon Alexa. Currently, these intelligent assistants do not maintain a lengthy information exchange. They do not track context or present information suitable for an audio-only channel, and do not interact with the user in a multi-turn conversation. Understanding how users would interact with such an audio-only interaction system in multi-turn information seeking dialogues, and what users expect from these new systems, are unexplored in search settings. In particular, the knowledge on how to present search results over an audio-only channel and which interactions take place in this new search paradigm is crucial to incorporate while producing usable systems [9, 2, 8]. Thus, constructing insight into the conversational structure of information seeking processes provides researchers and developers opportunities to build better systems while creating a research agenda and directions for future advancements in Spoken Conversational Search (SCS). Such insight has been identified as crucial in the growing SCS area.
At the moment, limited understanding has been acquired for SCS, for example, how the components interact, how information should be presented, or how task complexity impacts the interactivity or discourse behaviours. We aim to address these knowledge gaps. This thesis outlines the breadth of SCS and forms a manifesto advancing this highly interactive search paradigm with new research directions including prescriptive notions for implementing identified challenges [3].
We investigate SCS through quantitative and qualitative designs: (i) log and crowdsourcing experiments investigating different interaction and results presentation styles [1, 6], and (ii) the creation and analysis of the first SCS dataset and annotation schema through designing and conducting an observational study of information seeking dialogues [11, 5, 7]. We propose new research directions and design recommendations based on the triangulation of three different datasets and methods: the log analysis to identify practical challenges and limitations of existing systems while informing our future observational study; the crowdsourcing experiment to validate a new experimental setup for future search engine results presentation investigations; and the observational study to establish the SCS dataset (SCSdata), form the first Spoken Conversational Search Annotation Schema (SCoSAS), and study interaction behaviours for different task complexities.
Our principle contributions are based on our observational study for which we developed a novel methodology utilising a qualitative design [10]. We show that existing information seeking models may be insufficient for the new SCS search paradigm because they inadequately capture meta-discourse functions and the system's role as an active agent. Thus, the results indicate that SCS systems have to support the user through discourse functions and be actively involved in the users' search process. This suggests that interactivity between the user and system is necessary to overcome the increased complexity which has been imposed upon the user and system by the constraints of the audio-only communication channel [4]. We then present the first schematic model for SCS which is derived from the SCoSAS through the qualitative analysis of the SCSdata. In addition, we demonstrate the applicability of our dataset by investigating the effect of task complexity on interaction and discourse behaviour. Lastly, we present SCS design recommendations and outline new research directions for SCS.
The implications of our work are practical, conceptual, and methodological. The practical implications include the development of the SCSdata, the SCoSAS, and SCS design recommendations. The conceptual implications include the development of a schematic SCS model which identifies the need for increased interactivity and pro-activity to overcome the audio-imposed complexity in SCS. The methodological implications include the development of the crowdsourcing framework, and techniques for developing and analysing SCS datasets. In summary, we believe that our findings can guide researchers and developers to help improve existing interactive systems which are less constrained, such as mobile search, as well as more constrained systems such as SCS systems.

References

[1]
J. R. Trippas. Spoken conversational search: Information retrieval over a speech-only communication channel. In SIGIR, page 1067, 2015.
[2]
J. R. Trippas. Spoken conversational search: Speech-only interactive information retrieval. In CHIIR, pages 373--375, 2016.
[3]
J. R. Trippas. Spoken Conversational Search: Audio-only Interactive Information Retrieval. PhD thesis, RMIT, Melbourne, 2019.
[4]
J. R. Trippas, D. Spina, L. Cavedon, H. Joho, and M. Sanderson. Informing the design of spoken conversational search. In CHIIR, pages 32--41, 2018.
[5]
J. R. Trippas, D. Spina, L. Cavedon, and M. Sanderson. A conversational search transcription protocol and analysis. In CAIR'17 at SIGIR'17, 2017. 5 pages.
[6]
J. R. Trippas, D. Spina, L. Cavedon, and M. Sanderson. Crowdsourcing user preferences and query judgements for speech-only search. In CAIR'17 at SIGIR'17, 2017. 3 pages.
[7]
J. R. Trippas, D. Spina, L. Cavedon, and M. Sanderson. How do people interact in conversational speech-only search tasks: A preliminary analysis. In CHIIR, pages 325--328, 2017.
[8]
J. R. Trippas, D. Spina, M. Sanderson, and L. Cavedon. Results presentation methods for a spoken conversational search system. In NWSearch'15 at CIKM'15, pages 13--15, 2015.
[9]
J. R. Trippas, D. Spina, M. Sanderson, and L. Cavedon. Towards understanding the impact of length in web search result summaries over a speech-only communication channel. In SIGIR, pages 991--994, 2015.
[10]
J. R. Trippas, D. Spina, P. Thomas, H. Joho, M. Sanderson, and L. Cavedon. Towards a model for spoken conversational search. Information Processing & Management, 2019. (Accepted).
[11]
J. R. Trippas and P. Thomas. Data sets for spoken conversational search. In BIIRRR'19 at CHIIR'19. CEUR-WS, pages 14--18, 2019.

Cited By

View all
  • (2022)How to Approach Ambiguous Queries in Conversational Search: A Survey of Techniques, Approaches, Tools, and ChallengesACM Computing Surveys10.1145/353496555:6(1-40)Online publication date: 7-Dec-2022
  • (2022)Information Need AwarenessProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531999(610-621)Online publication date: 6-Jul-2022
  • (2020)Evaluating Conversational Recommender Systems via User SimulationProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403202(1512-1520)Online publication date: 6-Jul-2020
  1. Spoken conversational search: audio-only interactive information retrieval

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM SIGIR Forum
    ACM SIGIR Forum  Volume 53, Issue 2
    December 2019
    125 pages
    ISSN:0163-5840
    DOI:10.1145/3458553
    Issue’s Table of Contents
    Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 March 2021
    Published in SIGIR Volume 53, Issue 2

    Check for updates

    Qualifiers

    • Research-article

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)30
    • Downloads (Last 6 weeks)1
    Reflects downloads up to 17 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2022)How to Approach Ambiguous Queries in Conversational Search: A Survey of Techniques, Approaches, Tools, and ChallengesACM Computing Surveys10.1145/353496555:6(1-40)Online publication date: 7-Dec-2022
    • (2022)Information Need AwarenessProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3477495.3531999(610-621)Online publication date: 6-Jul-2022
    • (2020)Evaluating Conversational Recommender Systems via User SimulationProceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining10.1145/3394486.3403202(1512-1520)Online publication date: 6-Jul-2020

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Media

    Figures

    Other

    Tables

    Share

    Share

    Share this Publication link

    Share on social media