research-article

Does the User Have A Theory of the Recommender? A Grounded Theory Study

Authors:

Mohammed Muheeb Ghori,

Arman Dehpanah,

Jonathan Gemmell,

Hamed Qahri-Saremi,

Bamshad MobasherAuthors Info & Claims

UMAP '22 Adjunct: Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization

Pages 167 - 174

https://doi.org/10.1145/3511047.3537680

Published: 04 July 2022 Publication History

Abstract

Recommender systems have gained widespread adoption in many web applications. Modern internet users experience daily interactions with recommender systems. Consequently, users, through these interactive experiences, have developed an inherent understanding of how recommender systems work, what their objectives are, and how the user might manipulate them. We describe this understanding as the Theory of the Recommender. In this paper, we explore the users’ perception and understanding of the recommender system in an empirical study using a grounded theory methodology. To that end, we draw on the cognitive theory of mind to propose a comprehensive theoretical framework that explains the users’ interpretation of the recommender system’s knowledge, reasoning, motivation, beliefs and attitudes. Our findings, based on individual in-depth interviews, suggest that users possess a sophisticated understanding of the recommender system’s behavior. Identifying the user’s understanding is a necessary step in evaluating their impact and improving recommender systems accordingly. Finally, we discuss the potential implications of such user knowledge on recommendation performance.

References

[1]

D Scott Brandt and Lorna Uden. 2003. Insight into mental models of novice Internet searchers. Commun. ACM 46, 7 (2003), 133–136.

Digital Library

[2]

John M Carroll and John C Thomas. 1982. Metaphor and the cognitive representation of computing systems. IEEE Transactions on systems, man, and cybernetics 12, 2(1982), 107–116.

[3]

Juliet Corbin and Anselm Strauss. 2014. Basics of qualitative research: Techniques and procedures for developing grounded theory. Sage publications.

[4]

Juliet M Corbin and Anselm Strauss. 1990. Grounded theory research: Procedures, canons, and evaluative criteria. Qualitative sociology 13, 1 (1990), 3–21.

[5]

Jérôme Dinet and Muneo Kitajima. 2011. ” Draw me the Web” impact of mental model of the web on information search performance of young users. In Proceedings of the 23rd Conference on l’Interaction Homme-Machine. 1–7.

Digital Library

[6]

Chris Frith and Uta Frith. 2005. Theory of mind. Current Biology 15, 17 (2005), R644–R645.

[7]

Muheeb Faizan Ghori, Arman Dehpanah, Jonathan Gemmell, Hamed Qahri-Saremi, and Bamshad Mobasher. 2019. Does the User Have A Theory of the Recommender? A Pilot Study. (2019).

[8]

Barney G Glaser and Anselm L Strauss. 2017. Discovery of grounded theory: Strategies for qualitative research. Routledge.

[9]

Alvin I Goldman 2012. Theory of mind. The Oxford handbook of philosophy of cognitive science (2012), 402–424.

[10]

Andrew John Hutchison, Lynne Halley Johnston, and Jeff David Breckon. 2010. Using QSR-NVivo to facilitate the development of a grounded theory project: an account of a worked example. International Journal of Social Research Methodology 13, 4(2010), 283–302.

[11]

Dietmar Jannach and Malte Ludewig. 2017. Determining characteristics of successful recommendations from log data: a case study. In Proceedings of the Symposium on Applied Computing. 1643–1648.

Digital Library

[12]

Natalie A Jones, Helen Ross, Timothy Lynam, Pascal Perez, and Anne Leitch. 2011. Mental models: an interdisciplinary synthesis of theory and methods. Ecology and Society 16, 1 (2011).

[13]

Hanna Kallio, Anna-Maija Pietilä, Martin Johnson, and Mari Kangasniemi. 2016. Systematic methodological review: developing a framework for a qualitative semi-structured interview guide. Journal of advanced nursing 72, 12 (2016), 2954–2965.

[14]

Todd Kulesza, Simone Stumpf, Margaret Burnett, and Irwin Kwan. 2012. Tell me more?: the effects of mental model soundness on personalizing an intelligent agent. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM, 1–10.

Digital Library

[15]

Todd Kulesza, Simone Stumpf, Margaret Burnett, Sherry Yang, Irwin Kwan, and Weng-Keen Wong. 2013. Too much, too little, or just right? Ways explanations impact end users’ mental models. In 2013 IEEE Symposium on Visual Languages and Human Centric Computing. IEEE, 3–10.

[16]

Clayton Lewis and Robert Mack. 1982. Learning to use a text processing system: Evidence from “thinking aloud” protocols. In Proceedings of the 1982 conference on Human factors in computing systems. 387–392.

Digital Library

[17]

Thao Ngo and Nicole Krämer. 2021. It’s Just a Recipe?—Comparing Expert and Lay User Understanding of Algorithmic Systems. (2021).

[18]

Thao Ngo, Johannes Kunkel, and Jürgen Ziegler. 2020. Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative Study. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 183–191.

Digital Library

[19]

Donald A Norman. 1981. Categorization of action slips.Psychological review 88, 1 (1981), 1.

[20]

Donald A Norman. 1981. A psychologist views human processing: Human errors and other phenomena suggest processing mechanisms. In Proceedings of the 7th international joint conference on Artificial intelligence-Volume 2. 1097–1101.

[21]

Donald A Norman. 1983. Design rules based on analyses of human error. Commun. ACM 26, 4 (1983), 254–258.

Digital Library

[22]

Donald A Norman. 2014. Some observations on mental models. Psychology Press.

[23]

Pearl Pu, Li Chen, and Rong Hu. 2011. A user-centric evaluation framework for recommender systems. In Proceedings of the fifth ACM conference on Recommender systems. 157–164.

Digital Library

[24]

Lingyun Qiu and Izak Benbasat. 2009. Evaluating anthropomorphic product recommendation agents: A social relationship perspective to designing information systems. Journal of management information systems 25, 4 (2009), 145–182.

Digital Library

[25]

David E Rumelhart and Donald A Norman. 1980. Analogical Processes in Learning. Technical Report, April 1979-August 1980.(1980).

[26]

David E Rumelhart and Donald A Norman. 1983. Representation in memory.(1983).

[27]

David E Rumelhart, Donald A Norman, and JR Anderson. 1981. Analogical Processes.Lawrence Erlbaum Associates Hillsdale, NJ.

[28]

Rashmi Sinha and Kirsten Swearingen. 2002. The role of transparency in recommender systems. In CHI’02 extended abstracts on Human factors in computing systems. 830–831.

[29]

Kacper Sokol and Peter Flach. 2020. One explanation does not fit all. KI-Künstliche Intelligenz 34, 2 (2020), 235–250.

[30]

Daniel W Turner III. 2010. Qualitative interview design: A practical guide for novice investigators. The qualitative report 15, 3 (2010), 754.

[31]

Michael D Williams, James D Hollan, and Albert L Stevens. 2014. Human reasoning about a simple physical system. In Mental models. Psychology Press, 139–162.

Cited By

Rezk ASimkute AVines JElsden CEvans MJones RLuger E(2024)User-Centric Tensions: Exploring Perceived Benefits and (Dis)comfort in Media PersonalisationProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685365(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685365
Rezk ASimkute ALuger EVines JElsden CEvans MJones R(2024)Agency Aspirations: Understanding Users' Preferences And Perceptions Of Their Role In Personalised News CurationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642634(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642634
Khan SIqbal MOsho OSingh KDerrick KNelson PLi LSidnam-Mauch EBannister NCaine KKnijnenburg B(2024)Teaching Middle Schoolers about the Privacy Threats of Tracking and Pervasive Personalization: A Classroom Intervention Using Design-Based ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642460(1-26)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642460
Show More Cited By

Does the User Have A Theory of the Recommender? A Grounded Theory Study
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals

Recommendations

Exploring Mental Models for Transparent and Controllable Recommender Systems: A Qualitative Study
UMAP '20: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization

While online content is personalized to an increasing degree, eg. using recommender systems (RS), the rationale behind personalization and how users can adjust it typically remains opaque. This was often observed to have negative effects on the user ...
User Personality and User Satisfaction with Recommender Systems

In this study, we show that individual users' preferences for the level of diversity, popularity, and serendipity in recommendation lists cannot be inferred from their ratings alone. We demonstrate that we can extract strong signals about individual ...
Acquiring User Information Needs for Recommender Systems
WI-IAT '13: Proceedings of the 2013 IEEE/WIC/ACM International Joint Conferences on Web Intelligence (WI) and Intelligent Agent Technologies (IAT) - Volume 03

Most recommender systems attempt to use collaborative filtering, content-based filtering or hybrid approach to recommend items to new users. Collaborative filtering recommends items to new users based on their similar neighbours, and content-based ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

UMAP '22 Adjunct: Adjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization

July 2022

409 pages

ISBN:9781450392327

DOI:10.1145/3511047

General Chairs:
Alejandro Bellogin
Universidad Autonoma de Madrid, Spain
,
Ludovico Boratto
University of Cagliari, Italy
,
Olga C. Santos
UNED, Spain
,
Program Chairs:
Liliana Ardissono
Universita di Torino, Italy
,
Bart Knijnenburg
Clemson University, United States

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 04 July 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

UMAP '22

Sponsor:

UMAP '22: 30th ACM Conference on User Modeling, Adaptation and Personalization

July 4 - 7, 2022

Barcelona, Spain

Acceptance Rates

Overall Acceptance Rate 162 of 633 submissions, 26%

Upcoming Conference

UMAP '25

Sponsor:
sigchi
sigchi

33rd ACM Conference on User Modeling, Adaptation and Personalization

June 16 - 19, 2025

New York City , NY , USA

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
336
Total Downloads

Downloads (Last 12 months)103
Downloads (Last 6 weeks)8

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Rezk ASimkute AVines JElsden CEvans MJones RLuger E(2024)User-Centric Tensions: Exploring Perceived Benefits and (Dis)comfort in Media PersonalisationProceedings of the 13th Nordic Conference on Human-Computer Interaction10.1145/3679318.3685365(1-13)Online publication date: 13-Oct-2024
https://dl.acm.org/doi/10.1145/3679318.3685365
Rezk ASimkute ALuger EVines JElsden CEvans MJones R(2024)Agency Aspirations: Understanding Users' Preferences And Perceptions Of Their Role In Personalised News CurationProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642634(1-16)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642634
Khan SIqbal MOsho OSingh KDerrick KNelson PLi LSidnam-Mauch EBannister NCaine KKnijnenburg B(2024)Teaching Middle Schoolers about the Privacy Threats of Tracking and Pervasive Personalization: A Classroom Intervention Using Design-Based ResearchProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642460(1-26)Online publication date: 11-May-2024
https://dl.acm.org/doi/10.1145/3613904.3642460
Chambers OCohen RGrossman MHebert LAwad E(2024) Mining User Study Data to Judge the Merit of a Model for Supporting User‐Specific Explanations of AI Systems Computational Intelligence10.1111/coin.7001540:6Online publication date: 17-Dec-2024
https://doi.org/10.1111/coin.70015
Nyathi RMckenzie SLi JGorur RMohan Doss R(2024)User perceptions of algorithmic persuasion in OTT platforms: A scoping review2024 IEEE International Symposium on Technology and Society (ISTAS)10.1109/ISTAS61960.2024.10732741(1-7)Online publication date: 18-Sep-2024
https://doi.org/10.1109/ISTAS61960.2024.10732741
Rezk A(2023)“Way too good and way beyond comfort”Proceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society10.1145/3600211.3604761(996-998)Online publication date: 8-Aug-2023
https://dl.acm.org/doi/10.1145/3600211.3604761
Smits ABartels EDetweiler Cvan Turnhout K(2023)Algorithmic Affordances in Recommender InterfacesHuman-Computer Interaction – INTERACT 202310.1007/978-3-031-42293-5_80(605-609)Online publication date: 28-Aug-2023
https://dl.acm.org/doi/10.1007/978-3-031-42293-5_80

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten