skip to main content
10.1145/3460231.3478859acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
extended-abstract

Do Users Appreciate Explanations of Recommendations? An Analysis in the Movie Domain

Published: 13 September 2021 Publication History

Abstract

In this paper, we provide insights into users’ needs regarding the inclusion of explanations in a movie recommender system. We have developed different variants of a movie recommender system prototype corresponding to different types of explanations and conducted an online user study to evaluate related explanations. The experimental results show that users do not always appreciate explanations. They want to see explanations when they are not satisfied with the recommended items. They expect to see explanations showing how well the recommended item meets their preferences. Moreover, explanation goals are interdependent and affect the overall satisfaction of users with the recommender system.

Supplementary Material

MP4 File (Video_final_V5.mp4)
In recommender systems, explanations can be sent to users together with the recommendations in order to explain why a specific item has been chosen for them. Up to now, it is still unclear if users always appreciate explanations of recommendations. In this video, we present the general idea of our work regarding users' needs to include explanations in recommender systems. In our work, we developed different variants of a movie recommender system prototype corresponding to different types of explanations and conducted an online user study to evaluate related explanations. The experimental results show that users do not always appreciate explanations. Users want to see explanations when they are not satisfied with the recommended items. They expect to see explanations showing how well the recommended item meets their preferences. Moreover, explanation goals are interdependent and affect the overall satisfaction of users with the recommender system.

References

[1]
B. Abdollahi and O. Nasraoui. 2017. Using Explainability for Constrained Matrix Factorization. In Proceedings of the Eleventh ACM Conference on Recommender Systems (Como, Italy) (RecSys ’17). Association for Computing Machinery, New York, NY, USA, 79–83.
[2]
P. O Boadi, L. Guoxin, A. A. Sai, and P. Antwi. 2017. A Critique of the Impact of Customer Dissatisfaction. In International Conference on Transformations and Innovations in Management (ICTIM ’17), Shanghai, China, X. Zheng (Ed.). Atlantis Press, Zhengdong New District, China, 559–567.
[3]
J. Cohen. 1988. Statistical Power Analysis for the Behavioral Sciences. Routledge, Thames, Oxfordshire, UK.
[4]
H.S.M. Cramer, V. Evers, S. Ramlal, M. van Someren, L. Rutledge, N. Stash, L.M. Aroyo, and B.J. Wielinga. 2008. The Effects of Transparency on Trust in and Acceptance of a Content-based Art Recommender. User Modeling and User-Adapted Interaction 18, 5 (2008), 455–496.
[5]
A. Felfernig, M. Atas, T.N.T. Tran, M. Stettinger, S. P. Erdeniz, and G. Leitner. 2017. An Analysis of Group Recommendation Heuristics for High- and Low-Involvement Items. In IEA/AIE(Lecture Notes in Computer Science, Vol. 10350). Springer, Arras, France, 335–344.
[6]
A. Felfernig, B. Gula, and E. Teppan. 2007. Knowledge-based Recommender Technologies for Marketing and Sales. International journal of pattern recognition and artificial intelligence 21, 2(2007), 333–354. I.
[7]
J. L. Herlocker, J. A. Konstan, and J. Riedl. 2000. Explaining Collaborative Filtering Recommendations. In Proceedings of the 2000 ACM Conference on Computer Supported Cooperative Work (Philadelphia, Pennsylvania, USA) (CSCW ’00). Association for Computing Machinery, New York, NY, USA, 241–250.
[8]
M. T. Hoeve, M. Heruer, D. Odijk, A. Schuth, M. Spitters, and M. Rijke. 2017. Do News Consumers Want Explanations for Personalized News Rankings. In The Workshop on Responsible Recommendation (Como, Italy) (FATREC ’07). ACM, New York, NY, USA, 1–7.
[9]
Y. Hou, N. Yang, Y. Wu, and P. S. Yu. 2019. Explainable Recommendation with Fusion of Aspect Information. World Wide Web 22, 1 (Jan. 2019), 221–240.
[10]
Cataldo Musto, Fedelucio Narducci, Pasquale Lops, Marco de Gemmis, and Giovanni Semeraro. 2018. Linked Open Data-based Explanations for Transparent Recommender Systems. International Journal of Human-Computer Studies 121 (03 2018).
[11]
J. Pallant. 2007. SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS for Windows Version 15 (3rded.). Open University Press, Milton Keynes, UK, USA.
[12]
A. Papadimitriou, P. Symeonidis, and Y. Manolopoulos. 2012. A Generalized Taxonomy of Explanations Styles for Traditional and Social Recommender Systems. Data Mining and Knowledge Discovery 24, 3 (2012), 555–583.
[13]
Z. Ren, S. Liang, P. Li, S. Wang, and M. de Rijke. 2017. Social Collaborative Viewpoint Regression with Explainable Recommendations. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining(Cambridge, United Kingdom) (WSDM ’17). Association for Computing Machinery, New York, NY, USA, 485–494.
[14]
B. Sarwar, G. Karypis, J. Konstan, and J. Riedl. 2001. Item-Based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web (Hong Kong, Hong Kong) (WWW ’01). Association for Computing Machinery, New York, NY, USA, 285–295.
[15]
R. Sinha and K. Swearingen. 2002. The Role of Transparency in Recommender Systems. In CHI ’02 Extended Abstracts on Human Factors in Computing Systems (Minneapolis, Minnesota, USA) (CHI EA ’02). Association for Computing Machinery, New York, NY, USA, 830–831.
[16]
P. Symeonidis, A. Nanopoulos, and Y. Manolopoulos. 2008. Justified Recommendations based on Content and Rating Data. In WebKDD Workshop on Web Mining and Web Usage Analysis. ACM, Las Vegas, Nevada, 2–14.
[17]
N. Tintarev. 2007. Explanations of Recommendations. In Proceedings of the 2007 ACM Conference on Recommender Systems (Minneapolis, MN, USA) (RecSys ’07). Association for Computing Machinery, New York, NY, USA, 203–206.
[18]
N. Tintarev and J. Masthoff. 2007. A Survey of Explanations in Recommender Systems. In Proceedings of the 2007 IEEE 23rd International Conference on Data Engineering Workshop(ICDEW ’07). IEEE Computer Society, Istanbul, Turkey, 801–810.
[19]
N. Tintarev and J. Masthoff. 2012. Evaluating the Effectiveness of Explanations for Recommender Systems: Methodological Issues and Empirical Studies on the Impact of Personalization. User Modelling and User-Adapted Interaction 22, 4-5 (Oct. 2012), 399–439.
[20]
N. Tintarev and J. Masthoff. 2015. Explaining Recommendations: Design and Evaluation. Springer US, Boston, MA, 353–382.
[21]
C. H. Tsai and P. Brusilovsky. 2018. Explaining Social Recommendations to Casual Users: Design Principles and Opportunities. In Proceedings of the 23rd International Conference on Intelligent User Interfaces Companion (Tokyo, Japan) (IUI ’18 Companion). Association for Computing Machinery, New York, NY, USA, Article 59, 2 pages.
[22]
J. Vig, S. Sen, and J. Riedl. 2009. Tagsplanations: explaining recommendations using tags. In IUI ’09: Proceedings of the 13th international conference on Intelligent user interfaces (Sanibel Island, Florida, USA). ACM, New York, NY, USA, 47–56.
[23]
B. Wang, M. Ester, J. Bu, and D. Cai. 2014. Who Also Likes It? Generating the Most Persuasive Social Explanations in Recommender Systems. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Québec City, Québec, Canada, Carla E. Brodley and Peter Stone (Eds.). AAAI Press, California 94303, USA, 173–179.
[24]
Y. Zhang and X. Chen. 2020. Explainable Recommendation: A Survey and New Perspectives. Foundations and Trends® in Information Retrieval 14, 1 (2020), 1–101.
[25]
W. X. Zhao, Y. Guo, Y. He, H. Jiang, Y. Wu, and X. Li. 2014. We Know What You Want to Buy: a Demographic-based System for Produc Recommendation on Microblogs. In The 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’14, New York, NY, USA - August 24 - 27, 2014, Sofus A. Macskassy, Claudia Perlich, Jure Leskovec, Wei Wang, and Rayid Ghani (Eds.). ACM, New York, NY, USA, 1935–1944.
[26]
L. Zheng, V. Noroozi, and P. S. Yu. 2017. Joint Deep Modeling of Users and Items Using Reviews for Recommendation. In Proceedings of the Tenth ACM International Conference on Web Search and Data Mining(Cambridge, United Kingdom) (WSDM ’17). Association for Computing Machinery, New York, NY, USA, 425–434.

Cited By

View all
  • (2025)Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems EvaluationACM Transactions on Recommender Systems10.1145/3716394Online publication date: 5-Feb-2025
  • (2024)On the Negative Perception of Cross-domain Recommendations and ExplanationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657735(2102-2113)Online publication date: 10-Jul-2024
  • (2023)Understanding the Effects of Personalized Recommender Systems on Political News Perceptions: A Comparison of Content-Based, Collaborative, and Editorial Choice-Based News Recommender SystemJournal of Broadcasting & Electronic Media10.1080/08838151.2023.220666267:3(294-322)Online publication date: 4-May-2023

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
RecSys '21: Proceedings of the 15th ACM Conference on Recommender Systems
September 2021
883 pages
ISBN:9781450384582
DOI:10.1145/3460231
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 September 2021

Check for updates

Author Tags

  1. Collaborative Filtering-based Explanation
  2. Content-based Explanation
  3. Demographic-based Explanation
  4. Explanations
  5. Feature-based Explanation
  6. Knowledge-based Explanation.
  7. Movie Recommender Systems

Qualifiers

  • Extended-abstract
  • Research
  • Refereed limited

Conference

RecSys '21: Fifteenth ACM Conference on Recommender Systems
September 27 - October 1, 2021
Amsterdam, Netherlands

Acceptance Rates

Overall Acceptance Rate 254 of 1,295 submissions, 20%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)4
Reflects downloads up to 08 Mar 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Whom do Explanations Serve? A Systematic Literature Survey of User Characteristics in Explainable Recommender Systems EvaluationACM Transactions on Recommender Systems10.1145/3716394Online publication date: 5-Feb-2025
  • (2024)On the Negative Perception of Cross-domain Recommendations and ExplanationsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657735(2102-2113)Online publication date: 10-Jul-2024
  • (2023)Understanding the Effects of Personalized Recommender Systems on Political News Perceptions: A Comparison of Content-Based, Collaborative, and Editorial Choice-Based News Recommender SystemJournal of Broadcasting & Electronic Media10.1080/08838151.2023.220666267:3(294-322)Online publication date: 4-May-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media