Skip to main content
SpringerLink
Log in

Understanding Voice Search Behavior: Review and Synthesis of Research

  • Conference paper
  • First Online:
HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence (HCII 2020)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12424))

Included in the following conference series:

Abstract

This article reviews the recent user-oriented studies on voice search interaction and aims to understand how users perform and perceive voice search. With a systematic data collection and screening process, twenty-seven publications were included. We found that the current studies sampled predominantly young and well-educated population. Four topics of interest in voice search were identified, i.e., query characteristics, query strategy, spoken conversational search, user’s perception. The result reveals the influences of voice-based modality on users’ search behavior and perceptions and the review concludes with potential directions of research on voice search interaction.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    www.covidence.org.

  2. 2.

    Some of the publications were based on the same experiment but reported different aspects of the findings. In this review, a “study” refers to an experiment and a “publication” refers to a retrieved publication.

References

  1. Hearst, M.A.: Search User Interfaces. Cambridge University Press, New York (2009)

    Book  Google Scholar 

  2. Schalkwyk, J., et al.: Your word is my command: google search by voice: a case study. In: Neustein, A. (ed.) Advances in Speech Recognition, pp. 61–90. Springer, Boston (2010). https://doi.org/10.1007/978-1-4419-5951-5_4s

    Chapter  Google Scholar 

  3. Wilcock, G.: WikiTalk: a spoken Wikipedia-based open-domain knowledge access system. In: Proceedings of the Workshop on Question Answering for Complex Domains. p. 57 (2010)

    Google Scholar 

  4. Lee, C.-H., Wang, S.-M., Chang, H.-C., Lee, H.-Y.: ODSQA: open-domain spoken question answering dataset. In: 2018 IEEE Spoken Language Technology Workshop (SLT), pp. 949–956. IEEE (2018)

    Google Scholar 

  5. Sahib, N.G., Tombros, A., Stockman, T.: A comparative analysis of the information-seeking behavior of visually impaired and sighted searchers. J. Am. Soc. Inf. Sci. 63, 377–391 (2012)

    Article  Google Scholar 

  6. Smith, A.L., Chaparro, B.S.: Smartphone text input method performance, usability, and preference with younger and older adults. Hum. Factors 57, 1015–1028 (2015)

    Article  Google Scholar 

  7. Vtyurina, A., Fourney, A., Morris, M.R., Findlater, L., White, R.W.: Bridging screen readers and voice assistants for enhanced eyes-free web search. In: Liu, L., White, R. (eds.) The World Wide Web Conference on – WWW 2019, pp. 3590–3594. ACM Press, New York (2019)

    Google Scholar 

  8. Oard, D.W.: Query by babbling: a research agenda. In: Proceedings of the First Workshop on Information and Knowledge Management for Developing Region - IKM4DR 2012, p. 17. ACM Press, New York (2012)

    Google Scholar 

  9. Trippas, J.R., Thomas, P., Spina, D., Joho, H.: Third International Workshop on Conversational Approaches to Information Retrieval (CAIR 2020): Full-Day Workshop at CHIIR 2020. Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, pp. 492–494. ACM, New York (2020)

    Google Scholar 

  10. Culpepper, J.S., Diaz, F., Smucker, M.D.: Research frontiers in information retrieval. SIGIR Forum. 52, 34–90 (2018)

    Article  Google Scholar 

  11. Jiang, J., Jeng, W., He, D.: How do users respond to voice input errors? Lexical and phonetic query reformulation in voice search. In: Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval – SIGIR 2013, p. 143. ACM Press, New York (2013)

    Google Scholar 

  12. Arguello, J., Choi, B., Capra, R.: Factors influencing users’ information requests. ACM Trans. Inf. Syst. 36, 1–37 (2018)

    Article  Google Scholar 

  13. Sa, N., Jenny Yuan, X.: Examining users’ partial query modification patterns in voice search. J. Assoc. Inf. Sci. Technol. 71(3), 251–263 (2019)

    Article  Google Scholar 

  14. Trippas, J.R., Spina, D., Cavedon, L., Joho, H., Sanderson, M.: Informing the design of spoken conversational search: perspective paper. In: Proceedings of the 2018 Conference on Human Information Interaction & Retrieval, CHIIR 2018, pp. 32–41. ACM Press, New York (2018)

    Google Scholar 

  15. Ingwersen, P., Järvelin, K.: The turn: Integration of information seeking and retrieval in context. Springer, Dordrecht (2005). https://doi.org/10.1007/1-4020-3851-8

    Book  MATH  Google Scholar 

  16. Ruthven, I., Kelly, D. (eds.): Interactive Information Seeking, Behaviour and Retrieval. Facet (2011)

    Google Scholar 

  17. Vakkari, P.: Task-based information searching. Ann. Rev. Info. Sci. Tech. 37, 413–464 (2005)

    Article  Google Scholar 

  18. Belkin, N.J., Oddy, R.N., Brooks, H.M.: Ask for information retrieval: part i. background and theory. J. Doc. 38, 61–71 (1982)

    Article  Google Scholar 

  19. Kelly, D.: Methods for evaluating interactive information retrieval systems with users. FNT Inf. Retrieval 3, 1–224 (2009)

    Article  Google Scholar 

  20. White, R.W.: Interactions with Search Systems. Cambridge University Press, Cambridge (2016)

    Book  Google Scholar 

  21. Liu, J., Shah, C.: Interactive IR User Study Design, Evaluation, and Reporting. Synthesis Lectures on Information Concepts, Retrieval, and Services, vol. 11, p. i–75 (2019)

    Google Scholar 

  22. Grimes, C., Tang, D., Russell, D.: Query logs alone are not enough. In: Presented at the WWW 2007 Workshop on Query Log Analysis: Social and Technological Changes (2007)

    Google Scholar 

  23. Akinnaso, F.N.: On the differences between spoken and written language. Lang. Speech 25, 97–125 (1982)

    Article  Google Scholar 

  24. Zoltan-Ford, E.: How to get people to say and type what computers can understand. Int. J. Man Mach. Stud. 34, 527–547 (1991)

    Article  Google Scholar 

  25. Bigot, L.L., Jamet, E., Rouet, J.-F.: Searching information with a natural language dialogue system: a comparison of spoken vs written modalities. Appl. Ergon. 35, 557–564 (2004)

    Article  Google Scholar 

  26. Roberts, C., Street, B.: Spoken and written language. In: Coulmas, F. (ed.) The Handbook of Sociolinguistics, pp. 168–186. Blackwell Publishing Ltd, Oxford (2017)

    Chapter  Google Scholar 

  27. Jurafsky, D., Ranganath, R., McFarland, D.: Extracting social meaning. In: Association for Computational Linguistics (2009)

    Google Scholar 

  28. Guy, I.: The characteristics of voice search: comparing spoken with typed-in mobile web search queries. ACM Trans. Inf. Syst. 36, 1–28 (2018)

    Article  Google Scholar 

  29. Yi, J., Maghoul, F.: Mobile search pattern evolution: the trend and the impact of voice queries. In: Proceedings of the 20th International Conference Companion on World Wide Web – WWW 2011, p. 165. ACM Press, New York (2011)

    Google Scholar 

  30. Kamvar, M., Beeferman, D.: Say what? Why users choose to speak their web queries. In: INTERSPEECH 2010. isca-speech.org (2010)

    Google Scholar 

  31. Ingwersen, P.: Search procedures in the library—analysed from the cognitive point of view. J. Doc. 38, 165–191 (1982)

    Article  Google Scholar 

  32. Belkin, N.J.: Cognitive models and information transfer. Soc. Sci. Inf. Stud. 4, 111–129 (1984)

    Google Scholar 

  33. Belkin, N.J., Cool, C., Stein, A., Thiel, U.: Cases, scripts, and information-seeking strategies: on the design of interactive information retrieval systems. Expert Syst. Appl. 9, 379–395 (1995)

    Article  Google Scholar 

  34. Belkin, N.J., Marchetti, P., Cool, C.: BRAQUE: design of an interface to support user interaction in information retrieval. Inf. Process. Manage. 29, 325–344 (1993)

    Article  Google Scholar 

  35. Stein, A., Thiel, U.: A conversational model of multimodal interaction. In: Proceedings of the 11th National Conference on Artificial Intelligence (AAAI 1993), pp. 283–288. AAAI Press/The MIT Press, Menlo Park (1993)

    Google Scholar 

  36. Sitter, S., Stein, A.: Modeling the illocutionary aspects of information-seeking dialogues. Inf. Process. Manage. 28, 165–180 (1992)

    Article  Google Scholar 

  37. Radlinski, F., Craswell, N.: A theoretical framework for conversational search. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval – CHIIR 2017, pp. 117–126. ACM Press, New York (2017)

    Google Scholar 

  38. Azzopardi, L., Dubiel, M., Halvey, M., Dalton, J.: Conceptualizing agent-human interactions during the conversational search process (2018)

    Google Scholar 

  39. Trippas, J.R., Spina, D., Thomas, P., Sanderson, M., Joho, H., Cavedon, L.: Towards a model for spoken conversational search. Inf. Process. Manage. 57, 102162 (2020)

    Article  Google Scholar 

  40. Wang, Y.-Y., Yu, D., Ju, Y.-C., Acero, A.: An introduction to voice search. IEEE Signal Process. Mag. 25, 28–38 (2008)

    Article  Google Scholar 

  41. Barnard, E., Schalkwyk, J., Heerden, C.V., Moreno, P.J.: Voice search for development (2010)

    Google Scholar 

  42. Lovato, S.B., Piper, A.M.: Young children and voice search: what we know from human-computer interaction research. Front. Psychol. 10, 8 (2019)

    Article  Google Scholar 

  43. Kocaballi, A.B., Laranjo, L., Coiera, E.: Understanding and measuring user experience in conversational interfaces. Interact. Comput. 31, 192–207 (2019)

    Article  Google Scholar 

  44. Laranjo, L., et al.: Conversational agents in healthcare: a systematic review. J. Am. Med. Inform. Assoc. 25, 1248–1258 (2018)

    Article  Google Scholar 

  45. Vaidyam, A.N., Wisniewski, H., Halamka, J.D., Kashavan, M.S., Torous, J.B.: Chatbots and conversational agents in mental health: a review of the psychiatric landscape. Can. J. Psychiatry (2019). https://doi.org/10.1177/0706743719828977

  46. Kumah-Crystal, Y.A., Pirtle, C.J., Whyte, H.M., Goode, E.S., Anders, S.H., Lehmann, C.U.: Electronic health record interactions through voice: a review. Appl. Clin. Inf. 9, 541–552 (2018)

    Article  Google Scholar 

  47. Shah, C., González-Ibáñez, R.: Evaluating the synergic effect of collaboration in information seeking. In: Proceedings of the 34th International ACM SIGIR Conference on Research and Development in Information – SIGIR 2011, p. 913. ACM Press, New York (2011)

    Google Scholar 

  48. Avula, S., Arguello, J., Capra, R., Dodson, J., Huang, Y., Radlinski, F.: Embedding search into a conversational platform to support collaborative search. In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, CHIIR 2019, pp. 15–23. ACM Press, New York (2019)

    Google Scholar 

  49. Du, H., Crestani, F.: Spoken versus written queries for mobile information access: an experiment on Mandarin Chinese. In: Su, K.-Y., Tsujii, J., Lee, J.-H., Kwong, O.Y. (eds.) IJCNLP 2004. LNCS (LNAI), vol. 3248, pp. 745–754. Springer, Heidelberg (2005). https://doi.org/10.1007/978-3-540-30211-7_79

    Chapter  Google Scholar 

  50. Crestani, F., Du, H.: Written versus spoken queries: a qualitative and quantitative comparative analysis. J. Am. Soc. Inf. Sci. 57, 881–890 (2006)

    Article  Google Scholar 

  51. Bigot, L.L., Rouet, J.-F., Jamet, E.: Effects of speech- and text-based interaction modes in natural language human-computer dialogue. Hum Fact. 49, 1045–1053 (2007)

    Article  Google Scholar 

  52. Bigot, L.L., Terrier, P., Amiel, V., Poulain, G., Jamet, E., Rouet, J.-F.: Effect of modality on collaboration with a dialogue system. Int. J. Hum Comput Stud. 65, 983–991 (2007)

    Article  Google Scholar 

  53. Bigot, L.L., Terrier, P., Jamet, E., Botherel, V., Rouet, J.-F.: Does textual feedback hinder spoken interaction in natural language? Ergonomics 53, 43–55 (2010)

    Article  Google Scholar 

  54. Jeng, W., Jiang, J., He, D.: Users’ perceived difficulties and corresponding reformulation strategies in Google voice search. J. Libr. Inf. Stud. 14, 25–39 (2016)

    Google Scholar 

  55. Trippas, J.R., Spina, D., Sanderson, M., Cavedon, L.: Results presentation methods for a spoken conversational search system. In: Proceedings of the First International Workshop on Novel Web Search Interfaces and Systems – NWSearch 2015, pp. 13–15. ACM Press, New York (2015)

    Google Scholar 

  56. Kiseleva, J., et al.: Understanding user satisfaction with intelligent assistants. In: Proceedings of the 2016 ACM on Conference on Human Information Interaction and Retrieval – CHIIR 2016, pp. 121–130. ACM Press, New York (2016)

    Google Scholar 

  57. Begany, G.M., Sa, N., Yuan, X.: Factors affecting user perception of a spoken language vs. textual search interface: a content analysis. Interact. Comput. 28(2), 170–180 (2016)

    Google Scholar 

  58. Yuan, X.J., Sa, N.: User query behaviour in different task types in a spoken language vs. textual interface: a Wizard of Oz experiment. In: ISIC, the Information Behaviour Conference, Zadar, Croatia, 20–23 September 2016 (2017). Part 2. Information Research, 22(1), paper isic1615. IRinformationrearch

    Google Scholar 

  59. Kang, J., Condiff, K., Chang, S., Konstan, J.A., Terveen, L., Harper, F.M.: Understanding how people use natural language to ask for recommendations. In: Proceedings of the Eleventh ACM Conference on Recommender Systems - RecSys 2017, pp. 229–237. ACM Press, New York (2017)

    Google Scholar 

  60. Thomas, P., McDuff, D., Czerwinski, M., Craswell, N.: MISC: a data set of information-seeking conversations. Presented at the August 2017

    Google Scholar 

  61. Thomas, P., Czerwinski, M., McDuff, D., Craswell, N., Mark, G.: Style and alignment in information-seeking conversation. In: Proceedings of the 2018 Conference on Human Information Interaction&Retrieval, CHIIR 2018, pp. 42–51. ACM Press, New York (2018)

    Google Scholar 

  62. McDuff, D., Thomas, P., Czerwinski, M., Craswell, N.: Multimodal analysis of vocal collaborative search: a public corpus and results. In: Proceedings of the 19th ACM International Conference on Multimodal Interaction, ICMI 2017, pp. 456–463. ACM Press, New York (2017)

    Google Scholar 

  63. Trippas, J.R., Spina, D., Cavedon, L., Sanderson, M.: How do people interact in conversational speech-only search tasks: a preliminary analysis. In: Proceedings of the 2017 Conference on Conference Human Information Interaction and Retrieval – CHIIR 2017, pp. 325–328. ACM Press, New York (2017)

    Google Scholar 

  64. Dubiel, M., Halvey, M., Azzopardi, L.: Investigating how conversational search agents affect user’s behaviour, performance and search experience. In: ACM SIGIR CAIR Workshop (CAIR 2018), New York, NY, USA. strathprints.strath.ac.uk (2018)

    Google Scholar 

  65. Kiesel, J., Bahrami, A., Stein, B., Anand, A., Hagen, M.: Toward voice query clarification. In: The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, pp. 1257–1260. ACM Press, New York (2018)

    Google Scholar 

  66. Ziman, R., Walsh, G.: Factors affecting seniors’ perceptions of voice-enabled user interfaces. In: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, CHI 2018, pp. 1–6. ACM Press, New York (2018)

    Google Scholar 

  67. Kiesel, J., Bahrami, A., Stein, B., Anand, A., Hagen, M.: Clarifying false memories in voice-based search. In: Proceedings of the 2019 Conference on Human Information Interaction and Retrieval, CHIIR 2019, pp. 331–335. ACM Press, New York (2019)

    Google Scholar 

  68. Dubiel, M., Halvey, M., Azzopardi, L., Anderson, D., Daronnat, S.: Conversational strategies: impact on search performance in a goal-oriented task. In: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval. ACM, New York (2020)

    Google Scholar 

  69. Kiesel, J., Lang, K., Wachsmuth, H., Hornecker, E., Stein, B.: Investigating expectations for voice-based and conversational argument search on the web. In: Proceedings of the 2020 Conference on Human Information Interaction and Retrieval, pp. 53–62. ACM, New York (2020)

    Google Scholar 

  70. Tannen, D.: Conversational Style: Analyzing Talk Among Friends. Oxford University Press, Oxford (2005)

    Google Scholar 

  71. Sanchiz, M., Chin, J., Chevalier, A., Fu, W.T., Amadieu, F., He, J.: Searching for information on the web: impact of cognitive aging, prior domain knowledge and complexity of the search problems. Inf. Process. Manage. 53, 281–294 (2017)

    Article  Google Scholar 

  72. Dommes, A., Chevalier, A., Lia, S.: The role of cognitive flexibility and vocabulary abilities of younger and older users in searching for information on the web. Appl. Cogn. Psychol. 25, 717–726 (2011)

    Article  Google Scholar 

  73. André, E., et al.: Conversational search. In: Dagstuhl Seminar 19461, vol. 9, pp. 34–83 (2020)

    Google Scholar 

  74. Thomas, P., Czerwinski, M., McDuff, D., Craswell, N.: Theories of conversation for conversational IR. In: Presented at the SIGIR 3th International Workshop on Conversational Approaches to Information Retrieval (CAIR 2020), March 2020

    Google Scholar 

  75. Wachsmuth, H., Stein, B., Ajjour, Y.: “PageRank” for argument relevance. In: Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, pp. 1117–1127. Association for Computational Linguistics, Stroudsburg (2017)

    Google Scholar 

  76. Easwara Moorthy, A., Vu, K.-P.L.: Privacy concerns for use of voice activated personal assistant in the public space. Int. J. Hum. Comput. Interact. 31, 307–335 (2015)

    Article  Google Scholar 

  77. Lau, J., Zimmerman, B., Schaub, F.: Alexa, are you listening? Proc. ACM Hum.-Comput. Interact. 2, 1–31 (2018)

    Article  Google Scholar 

  78. Hadian, M., Altuwaiyan, T., Liang, X., Li, W.: Privacy-preserving voice-based search over mHealth data. Smart Health. 12, 24–34 (2018)

    Article  Google Scholar 

Download references

Acknowledgments

The authors would like to appreciate the support from the librarians at Health Science Library at the University of North Carolina at Chapel Hill. This study is partially supported by the Carolina Health Informatics Program and the Laboratory of Applied Informatics Research at the University of North Carolina at Chapel Hill. The authors also thank the reviewers for thoughtful and thorough review.

Author information

Authors and Affiliations

  1. University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA

    Zhaopeng Xing & Javed Mostafa

  2. University at Albany - State University of New York, Albany, NY, 12203, USA

    Xiaojun Yuan

  3. Wuhan University, Wuhan, Hubei, China

    Dan Wu & Yeman Huang

Authors
  1. Zhaopeng Xing
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaojun Yuan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dan Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yeman Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Javed Mostafa
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhaopeng Xing .

Editor information

Editors and Affiliations

  1. University of Crete and Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Constantine Stephanidis

  2. The Open University of Japan, Chiba, Japan

    Masaaki Kurosu

  3. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  4. University of Central Florida, Orlando, FL, USA

    Lauren Reinerman-Jones

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xing, Z., Yuan, X., Wu, D., Huang, Y., Mostafa, J. (2020). Understanding Voice Search Behavior: Review and Synthesis of Research. In: Stephanidis, C., Kurosu, M., Degen, H., Reinerman-Jones, L. (eds) HCI International 2020 - Late Breaking Papers: Multimodality and Intelligence. HCII 2020. Lecture Notes in Computer Science(), vol 12424. Springer, Cham. https://doi.org/10.1007/978-3-030-60117-1_23

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60117-1_23

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60116-4

  • Online ISBN: 978-3-030-60117-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation