research-article

How to Recommend?: User Trust Factors in Movie Recommender Systems

Authors:

Shlomo Berkovsky,

Dan ConwayAuthors Info & Claims

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

Pages 287 - 300

https://doi.org/10.1145/3025171.3025209

Published: 07 March 2017 Publication History

Abstract

How much trust a user places in a recommender is crucial to the uptake of the recommendations. Although prior work established various factors that build and sustain user trust, their comparative impact has not been studied in depth. This paper presents the results of a crowdsourced study examining the impact of various recommendation interfaces and content selection strategies on user trust. It evaluates the subjective ranking of nine key factors of trust grouped into three dimensions and examines the differences observed with respect to users' personality traits.

References

[1]

I. Benbasat and W. Wang. Trust in and adoption of online recommendation agents. Journal of the Association for Information Systems, 6(3), 2005.

[2]

Y. Benjamini and Y. Hochberg. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society. Series B (Methodological), pages 289--300, 1995.

[3]

L. M. Collins, J. J. Dziak, and R. Li. Design of experiments with multiple independent variables: a resource management perspective on complete and reduced factorial designs. Psychological methods, 14(3):202, 2009.

[4]

P. T. Costa and R. R. McCrae. Four ways five factors are basic. Personality and individual differences, 13(6):653--665, 1992.

[5]

H. S. M. Cramer, V. Evers, S. Ramlal, M. van Someren, L. Rutledge, N. Stash, L. Aroyo, and B. J. Wielinga. The effects of transparency on trust in and acceptance of a content-based art recommender. User Modeling and User-Adapted Interaction, 18(5):455--496, 2008.

Digital Library

[6]

A. Dillon and C. Watson. User analysis in HCI the historical lessons from individual differences research. International Journal of Human-Computer Studies, 45(6):619--637, 1996.

Digital Library

[7]

A. Felfernig and B. Gula. An empirical study on consumer behavior in the interaction with knowledge-based recommender applications. In Proceedings of the International Conference on E-Commerce Technology, CEC, page 37, 2006.

Digital Library

[8]

A. N. Finnerty, B. Lepri, and F. Pianesi. Acquisition of personality. In Emotions and Personality in Personalized Services, pages 81--99. 2016.

[9]

S. D. Gosling, P. J. Rentfrow, and W. B. Swann. A very brief measure of the big-five personality domains. Journal of Research in personality, 37(6):504--528, 2003.

[10]

A. Gunawardana and G. Shani. Evaluating recommender systems. In Recommender Systems Handbook, pages 265--308. 2015.

[11]

K. A. Hoff and M. Bashir. Trust in automation: Integrating empirical evidence on factors that influence trust. Human Factors, 57(3):407--434, 2015.

[12]

D. Holliday, S. M. Wilson, and S. Stumpf. User trust in intelligent systems: A journey over time. In Proceedings of the International Conference on Intelligent User Interfaces, IUI, pages 164--168, 2016.

Digital Library

[13]

A. Jameson, M. C. Willemsen, A. Felfernig, M. de Gemmis, P. Lops, G. Semeraro, and L. Chen. Human decision making and recommender systems. In Recommender Systems Handbook, pages 611--648. 2015.

[14]

J. D. Johnson, J. Sanchez, A. D. Fisk, and W. A. Rogers. Type of automation failure: The effects on trust and reliance in automation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 48, pages 2163--2167, 2004.

[15]

S. Y. Komiak and I. Benbasat. The effects of personalization and familiarity on trust and adoption of recommendation agents. Management Information Systems Quarterly, pages 941--960, 2006.

Digital Library

[16]

J. D. Lee and N. Moray. Trust, self-confidence, and operators' adaptation to automation. International Journal of Human-Computer Studies, 40(1):153--184, 1994.

Digital Library

[17]

J. D. Lee and K. A. See. Trust in automation: Designing for appropriate reliance. Human Factors, 46(1):50--80, 2004.

[18]

S. Matz, Y. W. F. Chan, and M. Kosinski. Models of personality. In Emotions and Personality in Personalized Services, pages 35--54. 2016.

[19]

N. Moray, T. Inagaki, and M. Itoh. Adaptive automation, trust, and self-confidence in fault management of time-critical tasks. Journal of Experimental Psychology: Applied, 6(1):44, 2000.

[20]

S. Nowak and S. M. Rüger. How reliable are annotations via crowdsourcing: a study about inter-annotator agreement for multi-label image annotation. In Proceedings of the International Conference on Multimedia Information Retrieval, MIR, pages 557--566, 2010.

Digital Library

[21]

J. O'Donovan and B. Smyth. Trust in recommender systems. In Proceedings of the International Conference on Intelligent User Interfaces, IUI, pages 167--174, 2005.

Digital Library

[22]

U. Panniello, M. Gorgoglione, and A. Tuzhilin. Research note In CARSs we trust: How context-aware recommendations affect customers' trust and other business performance measures of recommender systems. Information Systems Research, 27(1):182--196, 2016.

[23]

P. Pu and L. Chen. Trust building with explanation interfaces. In Proceedings of the International Conference on Intelligent User Interfaces, IUI, pages 93--100, 2006.

Digital Library

[24]

P. Pu, L. Chen, and R. Hu. A user-centric evaluation framework for recommender systems. In Proceedings of the ACM Conference on Recommender Systems, RecSys, pages 157--164, 2011.

Digital Library

[25]

W. B. Rouse. Adaptive aiding for human/computer control. Human Factors, 30(4):431--443, 1988.

Digital Library

[26]

A. Said, S. Berkovsky, E. W. D. Luca, and J. Hermanns. Challenge on context-aware movie recommendation: Camra2011. In Proceedings of the ACM Conference on Recommender Systems, RecSys, pages 385--386, 2011.

Digital Library

[27]

J. Sauer, A. Chavaillaz, and D. Wastell. Experience of automation failures in training: effects on trust, automation bias, complacency and performance. Ergonomics, pages 1--14, 2015.

[28]

K. E. Schaefer and D. R. Scribner. Individual differences, trust, and vehicle autonomy a pilot study. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, volume 59, pages 786--790, 2015.

[29]

J. Schrammel, C. Köffel, and M. Tscheligi. Personality traits, usage patterns and information disclosure in online communities. In Proceedings of the British Computer Society Conference on Human-Computer Interaction, BCS-HCI, pages 169--174, 2009.

Digital Library

[30]

C. L. Scott. Interpersonal trust: A comparison of attitudinal and situational factors. Human Relations, 33(11):805--812, 1980.

[31]

G. Shani, L. Rokach, B. Shapira, S. Hadash, and M. Tangi. Investigating confidence displays for top-N recommendations. Journal of the Association for Information Science and Technology, 64(12):2548--2563, 2013.

[32]

R. R. Sinha and K. Swearingen. Comparing recommendations made by online systems and friends. In Proceedings of the Workshop on Personalisation and Recommender Systems in Digital Libraries, 2001.

[33]

K. Swearingen and R. Sinha. Interaction design for recommender systems. In Designing Interactive Systems, volume 6, pages 312--334, 2002.

[34]

N. Tintarev and J. Masthoff. Explaining recommendations: Design and evaluation. In Recommender Systems Handbook, pages 353--382. 2015.

[35]

M. Tkalcic and L. Chen. Personality and recommender systems. In Recommender Systems Handbook, pages 715--739. 2015.

[36]

P. Victor, M. D. Cock, and C. Cornelis. Trust and recommendations. In Recommender Systems Handbook, pages 645--675. 2011.

[37]

A. Vinciarelli and G. Mohammadi. A survey of personality computing. IEEE Transactions on Affective Computing, 5(3):273--291, 2014.

[38]

Y. D. Wang and H. H. Emurian. An overview of online trust: Concepts, elements, and implications. Computers in Human Behavior, 21(1):105--125, 2005.

[39]

W. Wu, L. Chen, and L. He. Using personality to adjust diversity in recommender systems. In Proceedings of the Conference on Hypertext and Social Media, HT, pages 225--229, 2013.

Digital Library

[40]

S. Xiao and I. Benbasat. The formation of trust and distrust in recommendation agents in repeated interactions: a process-tracing analysis. In Proceedings of the International Conference on Electronic Commerce, ICEC, pages 287--293, 2003.

Digital Library

[41]

K. H. Yoo, U. Gretzel, and M. Zanker. Source factors in recommender system credibility evaluation. In Recommender Systems Handbook, pages 689--714. 2015.

[42]

K. Yu, S. Berkovsky, D. Conway, R. Taib, J. Zhou, and F. Chen. Trust and reliance based on system accuracy. In Proceedings of the Conference on User Modeling Adaptation and Personalization, UMAP, pages 223--227, 2016.

Digital Library

[43]

K. Yu, S. Berkovsky, R. Taib, D. Conway, J. Zhou, and F. Chen. User trust dynamics: An investigation driven by differences in system performance. In Proceedings of the International Conference on Intelligent User Interfaces, IUI, 2017.

Digital Library

Cited By

Hammann DWouters M(2025)Explainability Versus Accuracy of Machine Learning Models: The Role of Task Uncertainty and Need for Interaction with the Machine Learning ModelEuropean Accounting Review10.1080/09638180.2025.2463961(1-34)Online publication date: 25-Feb-2025
https://doi.org/10.1080/09638180.2025.2463961
Li DLiu ZZhang QKou YLiu TQu H(2025)Integrating User Sentiment and Behavior for Explainable RecommendationWeb and Big Data. APWeb-WAIM 2024 International Workshops10.1007/978-981-96-0055-7_12(135-148)Online publication date: 31-Jan-2025
https://doi.org/10.1007/978-981-96-0055-7_12
Li YDeng XHu XLiu J(2024)The Effects of E-Commerce Recommendation System Transparency on Consumer Trust: Exploring Parallel Multiple Mediators and a ModeratorJournal of Theoretical and Applied Electronic Commerce Research10.3390/jtaer1904012619:4(2630-2649)Online publication date: 1-Oct-2024
https://doi.org/10.3390/jtaer19040126
Show More Cited By

Index Terms

How to Recommend?: User Trust Factors in Movie Recommender Systems
1. Information systems
  1. Information retrieval

Recommendations

Let Me Explain: Impact of Personal and Impersonal Explanations on Trust in Recommender Systems
CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Trust in a Recommender System (RS) is crucial for its overall success. However, it remains underexplored whether users trust personal recommendation sources (i.e. other humans) more than impersonal sources (i.e. conventional RS), and, if they do, ...
A Cross-Cultural Analysis of Trust in Recommender Systems
UMAP '18: Proceedings of the 26th Conference on User Modeling, Adaptation and Personalization

User system trust is critical to the uptake of recommendations, and several factors of trust have been identified and compared. In this paper we present a cross-cultural, crowdsourced study examining user perceptions of nine factors of trust and link ...
Putting Users in Control of their Recommendations
RecSys '15: Proceedings of the 9th ACM Conference on Recommender Systems

The essence of a recommender system is that it can recommend items personalized to the preferences of an individual user. But typically users are given no explicit control over this personalization, and are instead left guessing about how their actions ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '17: Proceedings of the 22nd International Conference on Intelligent User Interfaces

March 2017

654 pages

ISBN:9781450343480

DOI:10.1145/3025171

General Chairs:
George A. Papadopoulos
The University of Cyprus, Cyprus
,
Tsvi Kuflik
The University of Haifa, Israel
,
Program Chairs:
Fang Chen
Data61, CSIRO Australia
,
Carlos Duarte
The University of Lisbon, Portugal
,
Wai-Tat Fu
The University of Illinois at Urbana-Champaign, USA

Copyright © 2017 ACM.

© 2017 Association for Computing Machinery. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of a national government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 March 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI'17

Sponsor:

IUI'17: 22nd International Conference on Intelligent User Interfaces

March 13 - 16, 2017

Limassol, Cyprus

Acceptance Rates

IUI '17 Paper Acceptance Rate 63 of 272 submissions, 23%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

62
Total Citations
View Citations
1,495
Total Downloads

Downloads (Last 12 months)110
Downloads (Last 6 weeks)12

Reflects downloads up to 08 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Hammann DWouters M(2025)Explainability Versus Accuracy of Machine Learning Models: The Role of Task Uncertainty and Need for Interaction with the Machine Learning ModelEuropean Accounting Review10.1080/09638180.2025.2463961(1-34)Online publication date: 25-Feb-2025
https://doi.org/10.1080/09638180.2025.2463961
Li DLiu ZZhang QKou YLiu TQu H(2025)Integrating User Sentiment and Behavior for Explainable RecommendationWeb and Big Data. APWeb-WAIM 2024 International Workshops10.1007/978-981-96-0055-7_12(135-148)Online publication date: 31-Jan-2025
https://doi.org/10.1007/978-981-96-0055-7_12
Li YDeng XHu XLiu J(2024)The Effects of E-Commerce Recommendation System Transparency on Consumer Trust: Exploring Parallel Multiple Mediators and a ModeratorJournal of Theoretical and Applied Electronic Commerce Research10.3390/jtaer1904012619:4(2630-2649)Online publication date: 1-Oct-2024
https://doi.org/10.3390/jtaer19040126
Lipinski PBalcer KLi XHandl J(2024)Explaining Session-based Recommendations using Grammatical EvolutionProceedings of the Genetic and Evolutionary Computation Conference Companion10.1145/3638530.3664156(1590-1597)Online publication date: 14-Jul-2024
https://dl.acm.org/doi/10.1145/3638530.3664156
Mao MZhang LLi JRivkin W(2024)My job is a game, and I am the owner: How gamification facilitates self-leadership for gig workersJournal of Business Research10.1016/j.jbusres.2024.114877183(114877)Online publication date: Oct-2024
https://doi.org/10.1016/j.jbusres.2024.114877
Ma JFeng X(2023)Analysing the Effects of Scenario-Based Explanations on Automated Vehicle HMIs from Objective and Subjective PerspectivesSustainability10.3390/su1601006316:1(63)Online publication date: 20-Dec-2023
https://doi.org/10.3390/su16010063
Falconnet ACoursaris CBeringer JVan Osch WSénécal SLéger P(2023)Improving User Experience with Recommender Systems by Informing the Design of Recommendation MessagesApplied Sciences10.3390/app1304270613:4(2706)Online publication date: 20-Feb-2023
https://doi.org/10.3390/app13042706
Jin YChen LCai WZhao X(2023)CRS-Que: A User-centric Evaluation Framework for Conversational Recommender SystemsACM Transactions on Recommender Systems10.1145/36315342:1(1-34)Online publication date: 2-Nov-2023
https://dl.acm.org/doi/10.1145/3631534
Hernandez-Bocanegra DZiegler J(2023)Explaining Recommendations through Conversations: Dialog Model and the Effects of Interface Type and Degree of InteractivityACM Transactions on Interactive Intelligent Systems10.1145/357954113:2(1-47)Online publication date: 12-Apr-2023
https://dl.acm.org/doi/10.1145/3579541
Lu HMa WWang YZhang MWang XLiu YChua TMa S(2023)User Perception of Recommendation Explanation: Are Your Explanations What Users Need?ACM Transactions on Information Systems10.1145/356548041:2(1-31)Online publication date: 25-Jan-2023
https://dl.acm.org/doi/10.1145/3565480
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten