research-article

Evaluating narrative-driven movie recommendations on Reddit

Authors:

Lukas Eberhard,

Denis HelicAuthors Info & Claims

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

Pages 1 - 11

https://doi.org/10.1145/3301275.3302287

Published: 17 March 2019 Publication History

Abstract

Recommender systems have become omni-present tools that are used by a wide variety of users in everyday life tasks, such as finding products in Web stores or online movie streaming portals. However, in situations where users already have an idea of what they are looking for (e.g., 'The Lord of the Rings', but in space with a dark vibe), most traditional recommender algorithms struggle to adequately address such a priori defined requirements. Therefore, users have built dedicated discussion boards to ask peers for suggestions, which ideally fulfill the stated requirements. In this paper, we set out to determine the utility of well-established recommender algorithms for calculating recommendations when provided with such a narrative. To that end, we first crowdsource a reference evaluation dataset from human movie suggestions. We use this dataset to evaluate the potential of five recommendation algorithms for incorporating such a narrative into their recommendations. Further, we make the dataset available for other researchers to advance the state of research in the field of narrative-driven recommendations. Finally, we use our evaluation dataset to improve not only our algorithmic recommendations, but also existing empirical recommendations of IMDb. Our findings suggest that the implemented recommender algorithms yield vastly different suggestions than humans when presented with the same a priori requirements. However, with carefully configured post-filtering techniques, we can outperform the baseline by up to 100%. This represents an important first step towards more refined algorithmic narrative-driven recommendations.

Supplementary Material

MP4 File (p1-eberhard.mp4)

Download
485.46 MB

References

[1]

Gediminas Adomavicius, Ramesh Sankaranarayanan, Shahana Sen, and Alexander Tuzhilin. 2005. Incorporating contextual information in recommender systems using a multidimensional approach. ACM Transactions on Information Systems (TOIS) 23, 1 (2005), 103--145.

Digital Library

[2]

Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and data engineering 17, 6 (2005), 734--749.

Digital Library

[3]

Gediminas Adomavicius and Alexander Tuzhilin. 2011. Context-Aware Recommender Systems. Springer US, Boston, MA, 217--253.

[4]

Gediminas Adomavicius, Alexander Tuzhilin, and Rong Zheng. 2011. REQUEST: A query language for customizing recommendations. Information Systems Research 22, 1 (2011), 99--117.

Digital Library

[5]

Chumki Basu, Haym Hirsh, William Cohen, et al. 1998. Recommendation as classification: Using social and content-based information in recommendation. In Aaai/iaai. 714--720.

Digital Library

[6]

Jason Michael Baumgartner. 2015. Reddit comment dataset. Website. (July 2015). https://www.reddit.com/r/datasets/comments/3bxlg7/i_have_every_publicly_available_reddit_comment.

[7]

Toine Bogers. 2015. Searching for Movies: An Exploratory Analysis of Movie-related Information Needs. iConference 2015 Proceedings (2015).

[8]

Toine Bogers and Marijn Koolen. 2017. Defining and Supporting Narrative-driven Recommendation. In Proceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 238--242.

Digital Library

[9]

Peter F. Brown, Peter V. deSouza, Robert L. Mercer, Vincent J. Della Pietra, and Jenifer C. Lai. 1992. Class-based N-gram Models of Natural Language. Comput. Linguist. 18, 4 (Dec. 1992), 467--479.

Digital Library

[10]

Konstantina Christakopoulou, Filip Radlinski, and Katja Hofmann. 2016. Towards Conversational Recommender Systems. In KDD. 815--824.

Digital Library

[11]

Christina Christakou, Spyros Vrettos, and Andreas Stafylopatis. 2007. A hybrid movie recommender system based on neural networks. International Journal on Artificial Intelligence Tools 16, 05 (2007), 771--792.

[12]

Christian Desrosiers and George Karypis. 2011. A comprehensive survey of neighborhood-based recommendation methods. Recommender systems handbook (2011), 107--144.

[13]

Simon Funk. 2006. Netflix update: Try this at home. (2006).

[14]

Sumit Ghosh, Manisha Mundhe, Karina Hernandez, and Sandip Sen. 1999. Voting for Movies: The Anatomy of a Recommender System. In Proceedings of the Third Annual Conference on Autonomous Agents (AGENTS '99). ACM, New York, NY, USA, 434--435.

Digital Library

[15]

Negar Hariri, Bamshad Mobasher, and Robin Burke. 2013. Query-driven Context Aware Recommendation. In Proceedings of the 7th ACM Conference on Recommender Systems (RecSys '13). ACM, New York, NY, USA, 9--16.

Digital Library

[16]

Zan Huang, Xin Li, and Hsinchun Chen. 2005. Link Prediction Approach to Collaborative Filtering. In Proceedings of the 5th ACM/IEEE-CS Joint Conference on Digital Libraries (JCDL '05). ACM, New York, NY, USA, 141--142.

Digital Library

[17]

Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM, 426--434.

Digital Library

[18]

Daniel Lamprecht, Florian Geigl, Tomas Karas, Simon Walk, Denis Helic, and Markus Strohmaier. 2015. Improving Recommender System Navigability Through Diversification: A Case Study of IMDb. In Proceedings of the 15th International Conference on Knowledge Technologies and Data-driven Business (i-KNOW '15). ACM, New York, NY, USA, Article 21, 8 pages.

Digital Library

[19]

Quoc Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. In International Conference on Machine Learning. 1188--1196.

Digital Library

[20]

Tariq Mahmood and Francesco Ricci. 2009. Improving Recommender Systems with Adaptive Conversational Strategies. In Proceedings of the 20th ACM Conference on Hypertext and Hypermedia (HT '09). ACM, New York, NY, USA, 73--82.

Digital Library

[21]

Harry Mak, Irena Koprinska, and Josiah Poon. 2003. Intimate: A web-based movie recommender using text categorization. In Web Intelligence, 2003. WI 2003. Proceedings. IEEE/WIC International Conference on. IEEE, 602--605.

Digital Library

[22]

Lorraine McGinty and James Reilly. 2011. On the evolution of critiquing recommenders. In Recommender Systems Handbook. Springer, 419--453.

[23]

Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013).

[24]

Kenta Oku, Shinsuke Nakajima, Jun Miyazaki, and Shunsuke Uemura. 2006. Context-aware SVM for context-dependent information recommendation. In Proceedings of the 7th international Conference on Mobile Data Management. IEEE Computer Society, 109.

Digital Library

[25]

Umberto Panniello, Alexander Tuzhilin, Michele Gorgoglione, Cosimo Palmisano, and Anto Pedone. 2009. Experimental comparison of pre-vs. post-filtering approaches in context-aware recommender systems. In Proceedings of the third ACM conference on Recommender systems. ACM, 265--268.

Digital Library

[26]

Arkadiusz Paterek. 2007. Improving regularized singular value decomposition for collaborative filtering. In Proceedings of KDD cup and workshop, Vol. 2007. 5--8.

[27]

Patrice Perny and Jean-Daniel Zucker. 2001. Preference-based search and machine learning for collaborative filtering: the "film-conseil" movie recommender system. Information, Interaction, Intelligence 1, 1 (2001), 9--48.

[28]

David Martin Powers. 2011. Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation. (2011).

[29]

Paul Resnick, Neophytos Iacovou, Mitesh Suchak, Peter Bergstrom, and John Riedl. 1994. GroupLens: An Open Architecture for Collaborative Filtering of Netnews. In Proceedings of the 1994 ACM Conference on Computer Supported Cooperative Work (CSCW '94). ACM, New York, NY, USA, 175--186.

Digital Library

[30]

Ruslan Salakhutdinov and Andriy Mnih. 2007. Probabilistic Matrix Factorization. In Nips, Vol. 1. 2--1.

Digital Library

[31]

Ruslan Salakhutdinov and Andriy Mnih. 2008. Bayesian probabilistic matrix factorization using Markov chain Monte Carlo. In Proceedings of the 25th international conference on Machine learning. ACM, 880--887.

Digital Library

[32]

Gerard Salton and Michael J McGill. 1986. Introduction to modern information retrieval. (1986).

Digital Library

[33]

Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based Collaborative Filtering Recommendation Algorithms. In Proceedings of the 10th International Conference on World Wide Web (WWW '01). ACM, New York, NY, USA, 285--295.

Digital Library

[34]

Paul Seitlinger, Dominik Kowald, Simone Kopeinik, Ilire Hasani-Mavriqi, Tobias Ley, and Elisabeth Lex. 2015. Attention Please! A Hybrid Resource Recommender Mimicking Attention-Interpretation Dynamics. In Proceedings of the 24th International Conference on World Wide Web (WWW '15 Companion). ACM, New York, NY, USA, 339--345.

Digital Library

[35]

Emine Yilmaz, Evangelos Kanoulas, and Javed A. Aslam. 2008. A Simple and Efficient Sampling Method for Estimating AP and NDCG. In Proceedings of the 31st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR '08). ACM, New York, NY, USA, 603--610.

Digital Library

Cited By

Cao TBogers T(2024)“If I Like BLANK, What Else Will I Like?”: Analyzing a Human Recommendation Community on RedditWisdom, Well-Being, Win-Win10.1007/978-3-031-57850-2_6(70-83)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1007/978-3-031-57850-2_6
Mysore SMccallum AZamani H(2023)Large Language Model Augmented Narrative Driven RecommendationsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608829(777-783)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608829
Wang YLee A(2023)Movie Account Recommendation on InstagramACM Transactions on Internet Technology10.1145/357985223:1(1-21)Online publication date: 13-Jan-2023
https://dl.acm.org/doi/10.1145/3579852
Show More Cited By

Index Terms

Evaluating narrative-driven movie recommendations on Reddit
1. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
      1. Recommender systems
  2. World Wide Web
    1. Web applications
      1. Crowdsourcing
    2. Web searching and information discovery
      1. Personalization

Recommendations

Tell Me What You Want: Embedding Narratives for Movie Recommendations
HT '20: Proceedings of the 31st ACM Conference on Hypertext and Social Media

Recommender systems are efficient exploration tools providing their users with valuable suggestions about items, such as products or movies. However, in scenarios where users have more specific ideas about what they are looking for (e.g., they provide ...
Personality, movie preferences, and recommendations
ASONAM '13: Proceedings of the 2013 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining

Personality is an important factor that influences people's decisions, actions, and tastes. While previous research has used surveys to establish a connection between personality and media preferences, to date there has been no research that connects ...
Making personalized movie recommendations for children
iiWAS '16: Proceedings of the 18th International Conference on Information Integration and Web-based Applications and Services

Multimedia have significant impact on the social and psychological development of children who are often explored to inappropriate materials, including movies that are either accessible online or through other multimedia channels. Even though not all ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

IUI '19: Proceedings of the 24th International Conference on Intelligent User Interfaces

March 2019

713 pages

ISBN:9781450362726

DOI:10.1145/3301275

General Chairs:
Wai-Tat Fu,
Shimei Pan,
Program Chairs:
Oliver Brdiczka,
Polo Chau,
Gaelle Calvary

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 17 March 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

IUI '19

Sponsor:

IUI '19: 24th International Conference on Intelligent User Interfaces

March 17 - 20, 2019

California, Marina del Ray

Acceptance Rates

IUI '19 Paper Acceptance Rate 71 of 282 submissions, 25%;

Overall Acceptance Rate 746 of 2,811 submissions, 27%

Upcoming Conference

IUI '25

Sponsor:
sigai
sigai

30th International Conference on Intelligent User Interfaces

March 24 - 27, 2025

Cagliari , Italy

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
752
Total Downloads

Downloads (Last 12 months)84
Downloads (Last 6 weeks)13

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Cao TBogers T(2024)“If I Like BLANK, What Else Will I Like?”: Analyzing a Human Recommendation Community on RedditWisdom, Well-Being, Win-Win10.1007/978-3-031-57850-2_6(70-83)Online publication date: 15-Apr-2024
https://dl.acm.org/doi/10.1007/978-3-031-57850-2_6
Mysore SMccallum AZamani H(2023)Large Language Model Augmented Narrative Driven RecommendationsProceedings of the 17th ACM Conference on Recommender Systems10.1145/3604915.3608829(777-783)Online publication date: 14-Sep-2023
https://dl.acm.org/doi/10.1145/3604915.3608829
Wang YLee A(2023)Movie Account Recommendation on InstagramACM Transactions on Internet Technology10.1145/357985223:1(1-21)Online publication date: 13-Jan-2023
https://dl.acm.org/doi/10.1145/3579852
Martina AMusto CIovine Ade Gemmis MNarducci FSemeraro GBellogin ABoratto LSantos OArdissono LKnijnenburg B(2022)A Virtual Assistant for the Movie Domain Exploiting Natural Language Preference Elicitation StrategiesAdjunct Proceedings of the 30th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3511047.3536407(7-12)Online publication date: 4-Jul-2022
https://dl.acm.org/doi/10.1145/3511047.3536407
Skenderi EHuhtamäki JStefanidis K(2021)Multi-Keyword Classification: A Case Study in Finnish Social Sciences Data ArchiveInformation10.3390/info1212049112:12(491)Online publication date: 25-Nov-2021
https://doi.org/10.3390/info12120491
Liu XWang GBhuiyan M(2021)Personalised context-aware re-ranking in recommender systemConnection Science10.1080/09540091.2021.199791534:1(319-338)Online publication date: 3-Nov-2021
https://doi.org/10.1080/09540091.2021.1997915
Eberhard LWalk SHelic DSalter ABarber JGadiraju U(2020)Tell Me What You WantProceedings of the 31st ACM Conference on Hypertext and Social Media10.1145/3372923.3404818(301-306)Online publication date: 13-Jul-2020
https://dl.acm.org/doi/10.1145/3372923.3404818
Wölbitsch MHasler TWalk SHelic D(2020)Mind the Gap: Exploring Shopping Preferences Across Fashion Retail ChannelsProceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization10.1145/3340631.3394866(257-265)Online publication date: 7-Jul-2020
https://dl.acm.org/doi/10.1145/3340631.3394866

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten