skip to main content
10.1145/2043932.2043957acmconferencesArticle/Chapter ViewAbstractPublication PagesrecsysConference Proceedingsconference-collections
research-article

Item popularity and recommendation accuracy

Published: 23 October 2011 Publication History

Abstract

Recommendations from the long tail of the popularity distribution of items are generally considered to be particularly valuable. On the other hand, recommendation accuracy tends to decrease towards the long tail. In this paper, we quantitatively examine this trade-off between item popularity and recommendation accuracy. To this end, we assume that there is a selection bias towards popular items in the available data. This allows us to define a new accuracy measure that can be gradually tuned towards the long tail. We show that, under this assumption, this measure has the desirable property of providing nearly unbiased estimates concerning recommendation accuracy. In turn, this also motivates a refinement for training collaborative-filtering approaches. In various experiments with real-world data, including a user study, empirical evidence suggests that only a small, if any, bias of the recommendations towards less popular items is appreciated by users.

References

[1]
C. Anderson. The Long Tail. Hyperion, New York, 2006.
[2]
J. Bennet and S. Lanning. The Netflix Prize. In Workshop at SIGKDD-07, ACM Conference on Knowledge Discovery and Data Mining, 2007.
[3]
W. G. Cochran. Sampling Techniques. Wiley, 1977.
[4]
P. Cremonesi, Y. Koren, and R. Turrin. Performance of recommender algorithms on top-N recommendation tasks. In ACM Conference on Recommender Systems, pages 39--46, 2010.
[5]
MovieLens data. homepage: http://www.grouplens.org/node/73, 2006.
[6]
S. Deerwester, S. Dumais, G. Furnas, R. Harshman, T. Landauer, K. Lochbaum, Lynn Streeter, et al. Latent semantic analysis / indexing. homepage: http://lsa.colorado.edu/.
[7]
S. Funk. Netflix update: Try this at home, 2006. http://sifter.org/simon/journal/20061211.html.
[8]
R. Groves, D. Dillman, J.L Eltinge, and R.J.A. Little. Survey Nonresponse. Wiley, 2002.
[9]
J. L. Herlocker, J. A. Konstan, L. G. Terveen, and J. T. Riedl. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems, 22:5--53, 2004.
[10]
Y. Hu, Y. Koren, and C. Volinsky. Collaborative filtering for implicit feedback datasets. In IEEE International Conference on Data Mining (ICDM), 2008.
[11]
R. Keshavan, A. Montanari, and S. Oh. Matrix completion from noisy entries. Journal of Machine Learning Research, 11:2057--78, 2010.
[12]
J. K. Kim and J. J. Kim. Nonresponse weighting adjustment using estimated response probability. The Canadian Journal of Statistics, 35:501--14, 2007.
[13]
Y. Koren. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In ACM Conference on Knowledge Discovery and Data Mining, pages 426--34, 2008.
[14]
L. Page, S. Brin, R. Motwani, and T. Winograd. The pagerank citation ranking: Bringing order to the web, 1999.
[15]
R. Pan, Y. Zhou, B. Cao, N. Liu, R. Lukose, M. Scholz, and Q. Yang. One-class collaborative filtering. In IEEE International Conference on Data Mining (ICDM), 2008.
[16]
A. Paterek. Improving regularized singular value decomposition for collaborative filtering. In KDDCup, 2007.
[17]
J. M. Robins, A. Rotnitzky, and L.P. Zhao. Estimation of regression coefficients when some regressors are not always observed. Journal of the American Statistical Association (JASA), 89:846--66, 1994.
[18]
P. R. Rosenbaum and D. B. Rubin. The central role of the propensity score in observational studies for causal effects. Biometrika, 70:41--55, 1983.
[19]
R. Salakhutdinov, A. Mnih, and G. Hinton. Restricted Boltzmann machines for collaborative filtering. In International Conference on Machine Learning (ICML), 2007.
[20]
R. Salakhutdinov and N. Srebro. Collaborative filtering in a non-uniform world: Learning with the weighted trace norm. In Advances in Neural Information Processing Systems 24 (NIPS), 2010.
[21]
C. F. Sarndal and S. Lundström. Estimation in Surveys with Nonresponse. Wiley, 2006.
[22]
C. F. Sarndal, B. Swensson, and J. Wretman. Model Assisted Survey Sampling. Springer, 1992.
[23]
G. Shani and A. Gunawardana. Evaluating recommendation systems. In Recommender Systems Handbook, pages 257--97. Springer, 2011.
[24]
H. Steck. Training and testing of recommender systems on data missing not at random. In ACM Conference on Knowledge Discovery and Data Mining, pages 713--22, 2010.
[25]
H. Steck and Y. Xin. A generalized probabilistic framework and its variants for training top-k recommender systems. In PRSAT Workshop at RecSys Conf., http://ceur-ws.org/#Vol-676, 2010.

Cited By

View all
  • (2025)Popularity Bias in Recommender Systems: The Search for Fairness in the Long TailInformation10.3390/info1602015116:2(151)Online publication date: 19-Feb-2025
  • (2025)CAFE+: Towards Compact, Adaptive, and Fast Embedding for Large-scale Online Recommendation ModelsACM Transactions on Information Systems10.1145/3713072Online publication date: 21-Jan-2025
  • (2025)Tiered matching model considering quality compatibility in two-sided marketsExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125835265:COnline publication date: 15-Mar-2025
  • Show More Cited By

Index Terms

  1. Item popularity and recommendation accuracy

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    RecSys '11: Proceedings of the fifth ACM conference on Recommender systems
    October 2011
    414 pages
    ISBN:9781450306836
    DOI:10.1145/2043932
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    In-Cooperation

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 23 October 2011

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tag

    1. recommender systems

    Qualifiers

    • Research-article

    Conference

    RecSys '11
    Sponsor:
    RecSys '11: Fifth ACM Conference on Recommender Systems
    October 23 - 27, 2011
    Illinois, Chicago, USA

    Acceptance Rates

    Overall Acceptance Rate 254 of 1,295 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)151
    • Downloads (Last 6 weeks)16
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Popularity Bias in Recommender Systems: The Search for Fairness in the Long TailInformation10.3390/info1602015116:2(151)Online publication date: 19-Feb-2025
    • (2025)CAFE+: Towards Compact, Adaptive, and Fast Embedding for Large-scale Online Recommendation ModelsACM Transactions on Information Systems10.1145/3713072Online publication date: 21-Jan-2025
    • (2025)Tiered matching model considering quality compatibility in two-sided marketsExpert Systems with Applications: An International Journal10.1016/j.eswa.2024.125835265:COnline publication date: 15-Mar-2025
    • (2024)A Novel Popularity Extraction Method Applied in Session-Based RecommendationTsinghua Science and Technology10.26599/TST.2023.901006129:4(971-984)Online publication date: Aug-2024
    • (2024)Investigating Characteristics of Media Recommendation Solicitation in r/ifyoulikeblankProceedings of the ACM on Human-Computer Interaction10.1145/36870418:CSCW2(1-23)Online publication date: 8-Nov-2024
    • (2024)Deep Causal Reasoning for RecommendationsACM Transactions on Intelligent Systems and Technology10.1145/365398515:4(1-25)Online publication date: 26-Mar-2024
    • (2024)Biased User History Synthesis for Personalized Long-Tail Item RecommendationProceedings of the 18th ACM Conference on Recommender Systems10.1145/3640457.3688141(189-199)Online publication date: 8-Oct-2024
    • (2024)Improving Recommendations for Non-Mainstream Users by Addressing Subjective Item ViewsAdjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3631700.3664916(35-39)Online publication date: 27-Jun-2024
    • (2024)User Perception of Fairness-Calibrated RecommendationsProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659558(78-88)Online publication date: 22-Jun-2024
    • (2024)Going Beyond Popularity and Positivity Bias: Correcting for Multifactorial Bias in Recommender SystemsProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657749(416-426)Online publication date: 10-Jul-2024
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media