skip to main content
10.1145/3640543.3645201acmconferencesArticle/Chapter ViewAbstractPublication PagesiuiConference Proceedingsconference-collections
research-article

Toward Faceted Skill Recommendation in Intelligent Personal Assistants

Published: 05 April 2024 Publication History

Abstract

Research continuously shows that, despite the wide range of skills developed for Intelligent Personal Assistants (IPAs), users tend to engage with only a small number of them. One reason for this discrepancy is the issue of skill discoverability, which is commonly addressed through conversational recommendations. Current recommendation strategies, however, are limited due to information asymmetry, lack of interactivity, and an underdeveloped understanding of appropriate grouping of available skills. In this paper, we explore opportunities for interactive faceted skill recommendations using voice interfaces. Through an open card sort user study and semi-structured interviews, we identify and describe five facets driving users’ natural grouping of IPA skills (Thematic, Procedural, Cross-system, Environmental, and Recipient), and demonstrate the need for simultaneous support of these facets. We then discuss the implications of these findings for advancing the discoverability of IPA skills through the design of interactive conversational recommendations.

References

[1]
Tawfiq Ammari, Jofish Kaye, Janice Y Tsai, and Frank Bentley. 2019. Music, Search, and IoT: How People (Really) Use Voice Assistants.ACM Trans. Comput. Hum. Interact. 26, 3 (2019), 17–1.
[2]
Sungeun An, Robert Moore, Eric Young Liu, and Guang-Jie Ren. 2021. Recipient Design for Conversational Agents: Tailoring Agent’s Utterance to User’s Knowledge. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–5.
[3]
Berkeley Andrus and Nancy Fulda. 2020. Immersive Gameplay via Improved Natural Language Understanding. In International Conference on the Foundations of Digital Games. 1–4.
[4]
Soodeh Atefi, Andrew Truelove, Matheus Rheinschmitt, Eduardo Almeida, Iftekhar Ahmed, and Amin Alipour. 2020. Examining user reviews of conversational systems: a case study of Alexa skills. arXiv preprint arXiv:2003.00919 (2020).
[5]
Senjuti Basu Roy, Haidong Wang, Gautam Das, Ullas Nambiar, and Mukesh Mohania. 2008. Minimum-effort driven dynamic faceted search in structured databases. In Proceedings of the 17th ACM conference on Information and knowledge management. 13–22.
[6]
Grace M Begany, Ning Sa, and Xiaojun Yuan. 2016. Factors affecting user perception of a spoken language vs. textual search interface: a content analysis. Interacting with computers 28, 2 (2016), 170–180.
[7]
Frank Bentley, Chris Luvogt, Max Silverman, Rushani Wirasinghe, Brooke White, and Danielle Lottridge. 2018. Understanding the long-term use of smart speaker assistants. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 3 (2018), 1–24.
[8]
Michael Bizzaco. 2021. The best Alexa skills of 2021. https://www.digitaltrends.com/home/best-alexa-skills/
[9]
James H Bradford. 1995. The human factors of speech-based interfaces: A research agenda. ACM SIGCHI Bulletin 27, 2 (1995), 61–67.
[10]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology. Qualitative research in psychology 3, 2 (2006), 77–101.
[11]
Raluca Budiu and Page Laubheimer. 2018. Intelligent Assistants Have Poor Usability: A User Study of Alexa, Google Assistant, and Siri. https://www.nngroup.com/articles/intelligent-assistant-usability/
[12]
Laura Burbach, Patrick Halbach, Nils Plettenberg, Johannes Nakayama, Martina Ziefle, and André Calero Valdez. 2019. " Hey, Siri"," Ok, Google"," Alexa". Acceptance-Relevant Factors of Virtual Voice-Assistants. In 2019 IEEE International Professional Communication Conference (ProComm). IEEE, 101–111.
[13]
Giovanni Campagna, Silei Xu, Mehrad Moradshahi, Richard Socher, and Monica S Lam. 2019. Genie: A generator of natural language semantic parsers for virtual assistant commands. In Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation. 394–410.
[14]
John M Carroll and Mary Beth Rosson. 1987. Paradox of the active user. In Interfacing thought: Cognitive aspects of human-computer interaction. 80–111.
[15]
Donald O Case and Lisa M Given. 2016. Looking for information: A survey of research on information seeking, needs, and behavior. (2016).
[16]
Konstantina Christakopoulou, Alex Beutel, Rui Li, Sagar Jain, and Ed H Chi. 2018. Q&R: A two-stage approach toward interactive recommendation. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 139–148.
[17]
Eric Corbett and Astrid Weber. 2016. What can I say? addressing user experience challenges of a mobile voice user interface for accessibility. In Proceedings of the 18th international conference on human-computer interaction with mobile devices and services. 72–82.
[18]
Benjamin R Cowan, Nadia Pantidi, David Coyle, Kellie Morrissey, Peter Clarke, Sara Al-Shehri, David Earley, and Natasha Bandeira. 2017. " What can i help you with?" infrequent users’ experiences of intelligent personal assistants. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services. 1–12.
[19]
Wisam Dakka, Rishabh Dayal, and Panagiotis G Ipeirotis. 2006. Automatic discovery of useful facet terms. In SIGIR Faceted Search Workshop. 18–22.
[20]
Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021. Advances and challenges in conversational recommender systems: A survey. AI Open 2 (2021), 100–126.
[21]
Ben Garland. 2021. The 30 Best Google Assistant Skills 2021. https://www.mymemory.co.uk/blog/the-30-best-google-assistant-skills-2019/
[22]
Eric Griffith and Rob Marvin. 2021. The Best Amazon Alexa Skills. https://www.pcmag.com/news/the-best-amazon-alexa-skills
[23]
Ramanathan Guha, Vineet Gupta, Vivek Raghunathan, and Ramakrishnan Srikant. 2015. User modeling for a personal assistant. In Proceedings of the Eighth ACM International Conference on Web Search and Data Mining. 275–284.
[24]
Ahmed Hassan Awadallah, Ranjitha Gurunath Kulkarni, Umut Ozertem, and Rosie Jones. 2015. Characterizing and predicting voice query reformulation. In Proceedings of the 24th ACM International on Conference on Information and Knowledge Management. 543–552.
[25]
Andrea Iovine. 2020. Conversational Agents for Recommender Systems. In Fourteenth ACM Conference on Recommender Systems. 758–763.
[26]
Manveer Kalirai and Anastasia Kuzminykh. 2022. What Can You Do For Me? The Discoverability of Intelligent Assistant Skills. In Adjunct Proceedings of the 2022 ACM International Joint Conference on Pervasive and Ubiquitous Computing and the 2022 ACM International Symposium on Wearable Computers. 57–59.
[27]
Laurent Karsenty. 2002. Shifting the design philosophy of spoken natural language dialogue: From invisible to transparent systems. International Journal of Speech Technology 5, 2 (2002), 147–157.
[28]
Philipp Kirschthaler, Martin Porcheron, and Joel E Fischer. 2020. What can I say? effects of discoverability in VUIs on task performance and user experience. In Proceedings of the 2nd Conference on Conversational User Interfaces. 1–9.
[29]
Laura Klein. 2015. Design for Voice Interfaces. O’Reilly Media.
[30]
Jonathan Koren, Yi Zhang, and Xue Liu. 2008. Personalized interactive faceted search. In Proceedings of the 17th international conference on World Wide Web. 477–486.
[31]
Kate Kozuch. 2021. The 30 best Alexa skills in 2021. https://www.tomsguide.com/round-up/best-alexa-skills
[32]
Brian Krisler and Richard Alterman. 2008. Training towards mastery: overcoming the active user paradox. In Proceedings of the 5th Nordic conference on Human-computer interaction: building bridges. 239–248.
[33]
Anastasia Kuzminykh and Edward Lank. 2016. People searched by people: Context-based selectiveness in online search. In Proceedings of the 2016 ACM Conference on Designing Interactive Systems. 749–760.
[34]
Guy Laban and Theo Araujo. 2020. The effect of personalization techniques in users’ perceptions of conversational recommender systems. In Proceedings of the 20th ACM International Conference on Intelligent Virtual Agents. 1–3.
[35]
Béatrice Lamche, Uwe Trottmann, and Wolfgang Wörndl. 2014. Active learning strategies for exploratory mobile recommender systems. In Proceedings of the 4th Workshop on Context-Awareness in Retrieval and Recommendation. 10–17.
[36]
Paul Lamkin. 2021. 101 Google Assistant commands: The best things to ask your Google Home. https://www.the-ambient.com/guides/best-google-assistant-commands-382
[37]
Federica Laricchia. 2022. Amazon Alexa: Skills Growth 2016-2019. https://www.statista.com/statistics/912856/amazon-alexa-skills-growth/
[38]
Josephine Lau, Benjamin Zimmerman, and Florian Schaub. 2018. Alexa, are you listening? Privacy perceptions, concerns and privacy-seeking behaviors with smart speakers. Proceedings of the ACM on Human-Computer Interaction 2, CSCW (2018), 1–31.
[39]
Ludovic Le Bigot, Loïc Caroux, Christine Ros, Agnès Lacroix, and Valérie Botherel. 2013. Investigating memory constraints on recall of options in interactive voice response system messages. Behaviour & Information Technology 32, 2 (2013), 106–116.
[40]
Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2020. Conversational Recommendation: Formulation, Methods, and Evaluation. In SIGIR 2020: 43rd international ACM SIGIR conference on Research and Development in Information Retrieval. ACM.
[41]
Wenqiang Lei, Xiangnan He, Yisong Miao, Qingyun Wu, Richang Hong, Min-Yen Kan, and Tat-Seng Chua. 2020. Estimation-action-reflection: Towards deep interaction between conversational and recommender systems. In Proceedings of the 13th International Conference on Web Search and Data Mining. 304–312.
[42]
Shijun Li, Wenqiang Lei, Qingyun Wu, Xiangnan He, Peng Jiang, and Tat-Seng Chua. 2021. Seamlessly unifying attributes and items: Conversational recommendation for cold-start users. ACM Transactions on Information Systems (TOIS) 39, 4 (2021), 1–29.
[43]
Wenda Li, Tan Yigitcanlar, Isil Erol, and Aaron Liu. 2021. Motivations, barriers and risks of smart home adoption: From systematic literature review to conceptual framework. Energy Research & Social Science 80 (2021), 102211.
[44]
Ewa Luger and Abigail Sellen. 2016. "Like Having a Really Bad PA" The Gulf between User Expectation and Experience of Conversational Agents. In Proceedings of the 2016 CHI conference on human factors in computing systems. 5286–5297.
[45]
Lydia Manikonda, Aditya Deotale, and Subbarao Kambhampati. 2018. What’s up with privacy? User preferences and privacy concerns in intelligent personal assistants. In Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society. 229–235.
[46]
Taylor Martin. 2021. The 50 Best Alexa Skills That Help Make Your Life Much Easier. https://www.cnet.com/home/smart-home/the-50-best-alexa-skills-that-help-make-your-life-so-much-easier/
[47]
Taylor Martin, David Priest, Dale Smith, and Andrew Gebhart. 2021. Every Google Assistant command for your Nest speaker or display available now. https://www.cnet.com/home/smart-home/google-assistant-commands-for-your-nest-speaker-or-display/
[48]
Pierrick Milhorat, Stephan Schlögl, Gérard Chollet, Jerome Boudy, Anna Esposito, and Gianni Pelosi. 2014. Building the next generation of personal digital assistants. In 2014 1st international conference on advanced technologies for signal and image processing (atsip). IEEE, 458–463.
[49]
Christine Murad, Cosmin Munteanu, Benjamin R. Cowan, and Leigh Clark. 2021. Finding a New Voice: Transitioning Designers from GUI to VUI Design. In CUI 2021-3rd Conference on Conversational User Interfaces. 1–12.
[50]
Chelsea M Myers, Anushay Furqan, and Jichen Zhu. 2019. The impact of user characteristics and preferences on performance with an unfamiliar voice user interface. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–9.
[51]
Don Norman. 2013. The design of everyday things: Revised and expanded edition. Basic books.
[52]
Scott Orgera. 2021. The 60 Most Useful Alexa Skills of 2021. https://www.lifewire.com/alexa-skills-4126799
[53]
Scott Orgera. 2021. The Top 100+ Google Assistant and Google Home Commands. https://www.lifewire.com/top-google-assistant-and-google-home-commands-4158256
[54]
Cathy Pearl. 2016. Designing voice user interfaces: Principles of conversational experiences. " O’Reilly Media, Inc.".
[55]
PwC. 2018. Consumer Intelligence Series: Prepare for the voice revolution. https://www.pwc.com/us/en/advisory-services/publications/consumer-intelligence-series/voice-assistants.pdf
[56]
Dimitrios Rafailidis and Yannis Manolopoulos. 2019. Can Virtual Assistants Produce Recommendations?. In Proceedings of the 9th International Conference on Web Intelligence, Mining and Semantics. 1–6.
[57]
Edison Research. 2022. The Infinite Dial 2022. https://www.edisonresearch.com/the-infinite-dial-2022/
[58]
Bradley James Rhodes and Pattie Maes. 2000. Just-in-time information retrieval agents. IBM Systems journal 39, 3.4 (2000), 685–704.
[59]
Xin Rong, Adam Fourney, Robin N Brewer, Meredith Ringel Morris, and Paul N Bennett. 2017. Managing uncertainty in time expressions for virtual assistants. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. 568–579.
[60]
Alex Sciuto, Arnita Saini, Jodi Forlizzi, and Jason I Hong. 2018. " Hey Alexa, What’s Up?" A Mixed-Methods Studies of In-Home Conversational Agent Usage. In Proceedings of the 2018 designing interactive systems conference. 857–868.
[61]
Shahin Sefati, Parsa Saadatpanah, Hassan Sayyadi, and Jan Neumann. 2018. Conversational content discovery via comcast X1 voice interface. In Proceedings of the 12th ACM Conference on Recommender Systems. 489–489.
[62]
Carmel Shavitt, Anastasia Kuzminykh, Itay Ridel, and Jessica R Cauchard. 2021. Naturally Together: A Systematic Approach for Multi-User Interaction With Natural Interfaces. Proceedings of the ACM on Human-Computer Interaction 5, CSCW2 (2021), 1–31.
[63]
Katie Sherwin. 2018. Make it Easy on the User: Designing for Discoverability within Mobile Apps. https://www.nngroup.com/articles/card-sorting-definition/
[64]
Ben Shneiderman. 2000. The limits of speech recognition. Commun. ACM 43, 9 (2000), 63–65.
[65]
Clemencia Siro, Mohammad Aliannejadi, and Maarten de Rijke. 2023. Understanding and Predicting User Satisfaction with Conversational Recommender Systems. ACM Transactions on Information Systems (2023).
[66]
James Stables. 2021. The best Alexa skills: our guide to apps for your Echo. https://www.the-ambient.com/guides/best-amazon-alexa-skills-187
[67]
Yueming Sun and Yi Zhang. 2018. Conversational recommender system. In The 41st international acm sigir conference on research & development in information retrieval. 235–244.
[68]
Daniel Tunkelang. 2009. Faceted search. Vol. 5. Morgan & Claypool Publishers.
[69]
Usability.gov. 2020. Card Sorting. https://www.usability.gov/how-to-and-tools/methods/card-sorting.html
[70]
André Calero Valdez and Martina Ziefle. 2019. The users’ perspective on the privacy-utility trade-offs in health recommender systems. International Journal of Human-Computer Studies 121 (2019), 108–121.
[71]
Ryen W White. 2018. Skill discovery in virtual assistants. Commun. ACM 61, 11 (2018), 106–113.
[72]
J Wilke, Fergus McInnes, Mervyn A Jack, and P Littlewood. 2007. Hidden menu options in automated human–computer telephone dialogues: dissonance in the user’s mental model. Behaviour & Information Technology 26, 6 (2007), 517–534.
[73]
Joshua Wissbroecker and F Maxwell Harper. 2018. Early lessons from a voice-only interface for finding movies. arXiv preprint arXiv:1808.09900 (2018).
[74]
Maria Wolters, Kallirroi Georgila, Johanna D Moore, Robert H Logie, Sarah E MacPherson, and Matthew Watson. 2009. Reducing working memory load in spoken dialogue systems. Interacting with Computers 21, 4 (2009), 276–287.
[75]
Amazon Staff Writer. 2023. The new Alexa design guide helps developers design skills that keep users coming back for more. https://developer.amazon.com/en-US/blogs/alexa/alexa-skills-kit/2023/03/alexa-design-guide-march-2023#: :text=The%20Alexa%20Design%20Guide%20helps, designer%20for%20Alexa%20Skills%20Kit.
[76]
Nicole Yankelovich. 1996. How do users know what to say?interactions 3, 6 (1996), 32–43.
[77]
Mark Yoshitake. 2023. New developer tools to build LLM-powered experiences with Alexa. https://developer.amazon.com/en-US/blogs/alexa/alexa-skills-kit/2023/09/alexa-llm-fall-devices-services-sep-2023
[78]
Yichi Zhang, Zhijian Ou, and Zhou Yu. 2020. Task-oriented dialog systems that consider multiple appropriate responses under the same context. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 9604–9611.

Cited By

View all
  • (2024)You Today, Better Tomorrow: Envisioning the Role of Conversation in Recommender Systems of the FutureProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665881(1-5)Online publication date: 8-Jul-2024

Index Terms

  1. Toward Faceted Skill Recommendation in Intelligent Personal Assistants

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      IUI '24: Proceedings of the 29th International Conference on Intelligent User Interfaces
      March 2024
      955 pages
      ISBN:9798400705083
      DOI:10.1145/3640543
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 05 April 2024

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. Intelligent personal assistants
      2. conversational recommendation
      3. discoverability
      4. skills
      5. voice interface

      Qualifiers

      • Research-article
      • Research
      • Refereed limited

      Conference

      IUI '24
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 746 of 2,811 submissions, 27%

      Upcoming Conference

      IUI '25

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)132
      • Downloads (Last 6 weeks)4
      Reflects downloads up to 20 Jan 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2024)You Today, Better Tomorrow: Envisioning the Role of Conversation in Recommender Systems of the FutureProceedings of the 6th ACM Conference on Conversational User Interfaces10.1145/3640794.3665881(1-5)Online publication date: 8-Jul-2024

      View Options

      Login options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      HTML Format

      View this article in HTML Format.

      HTML Format

      Media

      Figures

      Other

      Tables

      Share

      Share

      Share this Publication link

      Share on social media