Anastasiia Klimashevskaia
Mehdi Elahi
Dietmar Jannach
Christoph Trattner
ABSTRACT
Despite their proven various benefits, Recommender Systems can cause or amplify certain undesired effects. In this paper, we focus on Popularity Bias, i.e., the tendency of a recommender system to utilize the effect of recommending popular items to the user. Prior research has studied the negative impact of this type of bias on individuals and society as a whole and proposed various approaches to mitigate this in various domains. However, almost all works adopted offline methodologies to evaluate the effectiveness of the proposed approaches. Unfortunately, such offline simulations can potentially be rather simplified and unable to capture the full picture. To contribute to this line of research and given a particular lack of knowledge about how debiasing approaches work not only offline, but online as well, we present in this paper the results of user study on a national broadcaster movie streaming platform in Norway, i.e., TV 2, following the A/B testing methodology. We deployed an effective mitigation approach for popularity bias, called Calibrated Popularity (CP), and monitored its performance in comparison to the platform’s existing collaborative filtering recommendation approach as a baseline over a period of almost four months. The results obtained from a large user base interacting in real-time with the recommendations indicate that the evaluated debiasing approach can be effective in addressing popularity bias while still maintaining the level of user interest and engagement.
- Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher. 2017. Controlling popularity bias in learning-to-rank recommendation. In Proceedings of the 11th ACM Conference on Recommender Systems. 42–46.Google Scholar
Digital Library
- Himan Abdollahpouri, Masoud Mansoury, Robin Burke, and Bamshad Mobasher. 2019. The unfairness of popularity bias in recommendation. arXiv preprint arXiv:1907.13286 (2019).Google Scholar
- Himan Abdollahpouri, Masoud Mansoury, Robin Burke, Bamshad Mobasher, and Edward Malthouse. 2021. User-centered evaluation of popularity bias in recommender systems. In Proceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. 119–129.Google ScholarDigital Library
- Dirk Bollen, Bart P Knijnenburg, Martijn C Willemsen, and Mark Graus. 2010. Understanding choice overload in recommender systems. In Proceedings of the 4th ACM Conference on Recommender Systems. 63–70.Google ScholarDigital Library
- Paolo Cremonesi, Franca Garzotto, Roberto Pagano, and Massimo Quadrana. 2014. Recommending Without Short Head. In Proceedings of the 23rd International Conference on World Wide Web. 245–246.Google ScholarDigital Library
- Michael D Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All the cool kids, how do they fit in?: Popularity and demographic biases in recommender evaluation and effectiveness. In Conference on Fairness, Accountability and Transparency. PMLR, 172–186.Google Scholar
- Mehdi Elahi, Dietmar Jannach, Lars Skjærven, Erik Knudsen, Helle Sjøvaag, Kristian Tolonen, Øyvind Holmstad, Igor Pipkin, Eivind Throndsen, Agnes Stenbom, 2022. Towards responsible media recommendation. AI and Ethics (2022), 1–12.Google Scholar
- Farzad Eskandanian and Bamshad Mobasher. 2020. Using stable matching to optimize the balance between accuracy and diversity in recommendation. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. 71–79.Google ScholarDigital Library
- Daniel Fleder and Kartik Hosanagar. 2009. Blockbuster culture’s next rise or fall: The impact of recommender systems on sales diversity. Management science 55, 5 (2009), 697–712.Google Scholar
- Francisco Guíñez, Javier Ruiz, and María Ignacia Sánchez. 2021. Quantification of the Impact of Popularity Bias in Multi-stakeholder and Time-Aware Environments. In Advances in Bias and Fairness in Information Retrieval: Second International Workshop on Algorithmic Bias in Search and Recommendation, BIAS 2021, Lucca, Italy, April 1, 2021, Proceedings. Springer, 78–91.Google ScholarCross Ref
- Christian Hansen, Rishabh Mehrotra, Casper Hansen, Brian Brost, Lucas Maystre, and Mounia Lalmas. 2021. Shifting consumption towards diverse content on music streaming platforms. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 238–246.Google ScholarDigital Library
- Elvin Isufi, Matteo Pocchiari, and Alan Hanjalic. 2021. Accuracy-diversity trade-off in recommender systems via graph convolutions. Information Processing & Management 58, 2 (2021), 102459.Google ScholarCross Ref
- Dietmar Jannach, Lukas Lerche, Iman Kamehkhosh, and Michael Jugovac. 2015. What recommenders recommend: an analysis of recommendation biases and possible countermeasures. User Modeling and User-Adapted Interaction 25 (2015), 427–491.Google ScholarDigital Library
- Toshihiro Kamishima, Shotaro Akaho, Hideki Asoh, and Jun Sakuma. 2014. Correcting Popularity Bias by Enhancing Recommendation Neutrality. RecSys Posters 10 (2014).Google Scholar
- Anastasiia Klimashevskaia, Mehdi Elahi, Dietmar Jannach, Christoph Trattner, and Lars Skjærven. 2022. Mitigating Popularity Bias in Recommendation: Potential and Limits of Calibration Approaches. In Advances in Bias and Fairness in Information Retrieval: Third International Workshop, BIAS 2022, Stavanger, Norway, April 10, 2022, Revised Selected Papers. Springer, 82–90.Google Scholar
- Emanuel Lacic, Leon Fadljevic, Franz Weissenboeck, Stefanie Lindstaedt, and Dominik Kowald. 2022. What Drives Readership? An Online Study on User Interface Types and Popularity Bias Mitigation in News Article Recommendations. In European Conference on Information Retrieval. Springer, 172–179.Google Scholar
- Kibeom Lee and Kyogu Lee. 2015. Escaping your comfort zone: A graph-based recommender system for finding novel recommendations among relevant items. Expert Systems with Applications 42, 10 (2015), 4851–4858.Google ScholarDigital Library
- Masoud Mansoury, Himan Abdollahpouri, Mykola Pechenizkiy, Bamshad Mobasher, and Robin Burke. 2020. Feedback loop and bias amplification in recommender systems. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management (CIKM ’20). 2145–2148.Google ScholarDigital Library
- Yoon-Joo Park and Alexander Tuzhilin. 2008. The long tail of recommender systems and how to leverage it. In Proceedings of the 2008 ACM Conference on Recommender Systems. 11–18.Google ScholarDigital Library
- Sinan Seymen, Himan Abdollahpouri, and Edward C Malthouse. 2021. A Unified Optimization Toolbox for Solving Popularity Bias, Fairness, and Diversity in Recommender Systems.. In MORS@ RecSys.Google Scholar
- Harald Steck. 2011. Item popularity and recommendation accuracy. In Proceedings of the 5th ACM Conference on Recommender Systems. 125–132.Google ScholarDigital Library
- Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. 1791–1800.Google ScholarDigital Library
- Hongzhi Yin, Bin Cui, Jing Li, Junjie Yao, and Chen Chen. 2012. Challenging the Long Tail Recommendation. Proc. VLDB Endow. 5, 9 (2012), 896–907.Google ScholarDigital Library
- Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal intervention for leveraging popularity bias in recommendation. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 11–20.Google ScholarDigital Library
- Xiangyu Zhao, Zhendong Niu, and Wei Chen. 2013. Opinion-based collaborative filtering to solve popularity bias in recommender systems. In Database and Expert Systems Applications: 24th International Conference, DEXA 2013, Prague, Czech Republic, August 26-29, 2013. Proceedings, Part II 24. Springer, 426–433.Google ScholarCross Ref
- Reza Jafari Ziarani and Reza Ravanmehr. 2021. Serendipity in recommender systems: a systematic literature review. Journal of Computer Science and Technology 36 (2021), 375–396.Google ScholarDigital Library
Index Terms
- Evaluating The Effects of Calibrated Popularity Bias Mitigation: A Field Study
Recommendations
The Unfairness of Popularity Bias in Music Recommendation: A Reproducibility Study
Advances in Information RetrievalAbstractResearch has shown that recommender systems are typically biased towards popular items, which leads to less popular items being underrepresented in recommendations. The recent work of Abdollahpouri et al. in the context of movie recommendations ...
Investigating the impact of recommender systems on user-based and item-based popularity bias
AbstractRecommender Systems are decision support tools that adopt advanced algorithms in order to help users to find less-explored items that can be interesting for them. While recommender systems may offer a range of attractive benefits, they ...
Highlights- We measure reinforced recommendation-driven popularity bias on several algorithms.
User-centered Evaluation of Popularity Bias in Recommender Systems
UMAP '21: Proceedings of the 29th ACM Conference on User Modeling, Adaptation and PersonalizationRecommendation and ranking systems are known to suffer from popularity bias; the tendency of the algorithm to favor a few popular items while under-representing the majority of other items. Prior research has examined various approaches for mitigating ...
Comments