Jie Kang
Joseph A. Konstan
Loren Terveen
F. Maxwell Harper
ABSTRACT
The technical barriers for conversing with recommender systems using natural language are vanishing. Already, there are commercial systems that facilitate interactions with an AI agent. For instance, it is possible to say "what should I watch" to an Apple TV remote to get recommendations. In this research, we investigate how users initially interact with a new natural language recommender to deepen our understanding of the range of inputs that these technologies can expect. We deploy a natural language interface to a recommender system, we observe users' first interactions and follow-up queries, and we measure the differences between speaking- and typing-based interfaces. We employ qualitative methods to derive a categorization of users' first queries (objective, subjective, and navigation) and follow-up queries (refine, reformulate, start over). We employ quantitative methods to determine the differences between speech and text, finding that speech inputs are typically longer and more conversational.
Supplemental Material
Available for Download
zip
recs025.zip (8.3 KB)
# Understanding How People Use Natural Language to Ask for Recommendations: Query Dataset
- Andreas Bohm. 2004. TheoreticaI Coding: Text AnaIysis in Grounded Theory. In A companion to qualitative research. 270.Google Scholar
- Andrei Broder. 2002. A Taxonomy of Web Search. SIGIR Forum 36, 2 (Sept. 2002), 3--10. Google ScholarDigital Library
- Barbara L. Chalfonte, Robert S. Fish, and Robert E. Kraut. 1991. Expressive Richness: A Comparison of Speech and Text As Media for Revision. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '91). ACM, New York, NY, USA, 21--26. Google ScholarDigital Library
- Ed H. Chi, Peter Pirolli, Kim Chen, and James Pitkow. 2001. Using Information Scent to Model User Information Needs and Actions and the Web. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '01). ACM, New York, NY, USA, 490--497. Google ScholarDigital Library
- Lucas Colucci, Prachi Doshi, Kun-Lin Lee, Jiajie Liang, Yin Lin, Ishan Vashishtha, Jia Zhang, and Alvin Jude. 2016. Evaluating Item-Item Similarity Algorithms for Movies. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '16). ACM, New York, NY, USA, 2141--2147. Google ScholarDigital Library
- Fabio Crestani and Heather Du. 2006. Written Versus Spoken Queries: A Qualitative and Quantitative Comparative Analysis. J. Am. Soc. Inf. Sci. Technol. 57, 7 (May 2006), 881--890. Google ScholarDigital Library
- Jane Dye, Irene Schatz, Brian Rosenberg, and Susanne Coleman. 2000. Constant Comparison Method: A Kaleidoscope of Data. The Qualitative Report 4, 1 (Jan. 2000), 1--10.Google Scholar
- Mehmet H. Göker and Cynthia A. Thompson. 2000. Personalized Conversational Case-Based Recommendation. In Advances in Case-Based Reasoning, Enrico Blanzieri and Luigi Portinale (Eds.). Number 1898 in Lecture Notes in Computer Science. Springer Berlin Heidelberg, 99--111. Google ScholarDigital Library
- Ido Guy. 2016. Searching by Talking: Analysis of Voice Queries on Mobile Web Search. In Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '16). ACM, New York, NY, USA, 35--44. Google ScholarDigital Library
- Jonathan L. Herlocker, Joseph A. Konstan, Loren G. Terveen, and John T. Riedl. 2004. Evaluating Collaborative Filtering Recommender Systems. ACM Trans. Inf. Syst. 22, 1 (Jan. 2004), 5--53. Google ScholarDigital Library
- Bernard J. Jansen, Danielle L. Booth, and Amanda Spink. 2008. Determining the informational, navigational, and transactional intent of Web queries. Information Processing & Management 44, 3 (May 2008), 1251--1266. Google ScholarDigital Library
- Jiepu Jiang, Wei Jeng, and Daqing He. 2013. How Do Users Respond to Voice Input Errors? Lexical and Phonetic Query Reformulation in Voice Search. In Proceedings of the 36th International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '13). ACM, New York, NY, USA, 143--152. Google ScholarDigital Library
- Maryam Kamvar and Shumeet Baluja. 2006. A Large Scale Study of Wireless Search Behavior: Google Mobile Search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '06). ACM, New York, NY, USA, 701--709. Google ScholarDigital Library
- Jie Kang, Kyle Condiff, Shuo Chang, Joseph A. Konstan, Loren Terveen, and F. Maxwell Harper. 2017. Understanding How People Use Natural Language to Ask for Recommendations: Query Dataset. (June 2017). http://conservancy.umn.edu/handle/11299/188687 type: dataset. Google ScholarDigital Library
- Lorraine McGinty and James Reilly. 2011. On the Evolution of Critiquing Recommenders. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor (Eds.). Springer US, 419--453.Google Scholar
- Sean M. McNee, John Riedl, and Joseph A. Konstan. 2006. Making Recommendations Better: An Analytic Model for Human-recommender Interaction. In CHI '06 Extended Abstracts on Human Factors in Computing Systems (CHI EA '06). ACM, New York, NY, USA, 1103--1108. Google ScholarDigital Library
- Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed Representations of Words and Phrases and their Compositionality. In Advances in Neural Information Processing Systems 26, C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (Eds.). Curran Associates, Inc., 3111--3119. Google ScholarDigital Library
- Hendrik Müller, Aaron Sedley, and Elizabeth Ferrall-Nunge. 2014. Survey Research in HCI. In Ways of Knowing in HCI, Judith S. Olson and Wendy A. Kellogg (Eds.). Springer New York, 229--266.Google Scholar
- Peter Pirolli and Stuart Card. 1999. Information foraging. Psychological Review 106, 4 (1999), 643--675.Google ScholarCross Ref
- Pearl Pu, Li Chen, and Rong Hu. 2011. A User-centric Evaluation Framework for Recommender Systems. In Proceedings of the Fifth ACM Conference on Recommender Systems (RecSys '11). ACM, New York, NY, USA, 157--164. Google ScholarDigital Library
- Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to Recommender Systems Handbook. In Recommender Systems Handbook, Francesco Ricci, Lior Rokach, Bracha Shapira, and Paul B. Kantor (Eds.). Springer US, 1--35.Google Scholar
- Daniel E. Rose and Danny Levinson. 2004. Understanding User Goals in Web Search. In Proceedings of the 13th International Conference on World Wide Web (WWW '04). ACM, New York, NY, USA, 13--19. Google ScholarDigital Library
- Sherry Ruan, Jacob O. Wobbrock, Kenny Liou, Andrew Ng, and James Landay. 2016. Speech Is 3x Faster than Typing for English and Mandarin Text Entry on Mobile Devices. arXiv:1608.07323 {cs.HC} (Aug. 2016).Google Scholar
- Gery W. Ryan and H. Russell Bernard. 2003. Techniques to Identify Themes. Field Methods 15, 1 (Feb. 2003), 85--109.Google ScholarCross Ref
- J. Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative Filtering Recommender Systems. In The Adaptive Web, Peter Brusilovsky, Alfred Kobsa, and Wolfgang Nejdl (Eds.). Number 4321 in Lecture Notes in Computer Science. Springer Berlin Heidelberg, 291--324.Google Scholar
- B. Smyth, L. McGinty, J. Reilly, and K. McCarthy. 2004. Compound Critiques for Conversational Recommender Systems. In IEEE/WIC/ACM International Conference on Web Intelligence, 2004. WI 2004. Proceedings. 145--151. Google ScholarDigital Library
- Jaime Teevan, Christine Alvarado, Mark S. Ackerman, and David R. Karger. 2004. The Perfect Search Engine is Not Enough: A Study of Orienteering Behavior in Directed Search. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '04). ACM, New York, NY, USA, 415--422. Google ScholarDigital Library
- Cynthia A. Thompson, Mehmet H. Göker, and Pat Langley. 2004. A Personalized System for Conversational Recommendations. J. Artif. Int. Res. 21, 1 (March 2004), 393--428. Google ScholarDigital Library
- Jesse Vig, Shilad Sen, and John Riedl. 2012. The Tag Genome: Encoding Community Knowledge to Support Novel Interaction. ACM Trans. Interact. Intell. Syst. 2, 3 (Sept. 2012), 13:1--13:44. Google ScholarDigital Library
- Steve Whittaker, Julia Hirschberg, Brian Amento, Litza Stark, Michiel Bacchiani, Philip Isenhour, Larry Stead, Gary Zamchick, and Aaron Rosenberg. 2002. SCANMail: A Voicemail Interface That Makes Speech Browsable, Readable and Searchable. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '02). ACM, New York, NY, USA, 275--282. Google ScholarDigital Library
Index Terms
- Understanding How People Use Natural Language to Ask for Recommendations
Recommendations
Collaborative Filtering for people-to-people recommendation in online dating
A common perception is that online dating systems "match" people on the basis of profiles containing demographic and psychographic information and/or user interests. In contrast, product recommender systems are typically based on Collaborative Filtering,...
Putting Users in Control of their Recommendations
RecSys '15: Proceedings of the 9th ACM Conference on Recommender SystemsThe essence of a recommender system is that it can recommend items personalized to the preferences of an individual user. But typically users are given no explicit control over this personalization, and are instead left guessing about how their actions ...
Acquiring and Revising Preferences in a Critique-Based Mobile Recommender System
Personalized product recommendations, generated by e-commerce recommender systems, can provide helpful information and decision support when users are overwhelmed by product alternatives. Although many successful Web-based recommender systems have been ...
Comments