Skip to main content
SpringerLink
Log in

The Impact of a Popularity Punishing Hyperparameter on ItemKNN Recommendation Performance

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13981))

Included in the following conference series:

  • 1465 Accesses

Abstract

Collaborative filtering techniques have a tendency to amplify popularity biases present in the training data if no countermeasures are taken. The ItemKNN algorithm with conditional probability-inspired similarity function has a hyperparameter \(\alpha \) that allows one to counteract this popularity bias. In this work, we perform a deep dive into the effects of this hyperparameter in both online and offline experiments, with regard to both accuracy metrics and equality of exposure. Our experiments show that the hyperparameter can indeed counteract popularity bias in a dataset. We also find that there exists a trade-off between countering popularity bias and the quality of the recommendations: Reducing popularity bias too much results in a decrease in click-through rate, but some counteracting of popularity bias is required for optimal online performance.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Abdollahpouri, H., Burke, R., Mobasher, B.: Controlling popularity bias in learning-to-rank recommendation. In: Proceedings of the Eleventh ACM Conference on Recommender Systems, RecSys 2017, pp. 42–46. Association for Computing Machinery, New York (2017). https://doi.org/10.1145/3109859.3109912. ISBN 9781450346528

  2. Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B.: The unfairness of popularity bias in recommendation. In: CEUR Workshop Proceedings, vol. 2440 (2019). https://ceur-ws.org/Vol-2440/paper4.pdf

  3. Bambini, R., Cremonesi, P., Turrin, R.: A recommender system for an IPTV service provider: a real large-scale production environment. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 299–331. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_9

    Chapter  Google Scholar 

  4. Beel, J., Genzmehr, M., Langer, S., Nürnberger, A., Gipp, B.: A comparative analysis of offline and online evaluations and discussion of research paper recommender system evaluation. In: Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation, RepSys 2013, pp. 7–14. Association for Computing Machinery, New York (2013). https://doi.org/10.1145/2532508.2532511. ISBN 9781450324656

  5. Beel, J., Langer, S.: A comparison of offline evaluations, online evaluations and user studies in the context of research-paper recommender systems. In: Kapidakis, S., Mazurek, C., Werla, M. (eds.) Research and Advanced Technology for Digital Libraries, pp. 153–168. Springer Cham (2015). https://doi.org/10.1007/978-3-319-24592-8_12. ISBN 978-3-319-24592-8

  6. Castells, P., Hurley, N., Vargas, S.: Novelty and diversity in recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 603–646. Springer, New York (2022). https://doi.org/10.1007/978-1-0716-2197-4_16. ISBN 978-1-0716-2197-4

  7. Deshpande, M., Karypis, G.: Item-based top-n recommendation algorithms. ACM Trans. Inf. Syst. 22(1), 143–177 (2004). https://doi.org/10.1145/963770.963776. ISSN 1046–8188

    Article  Google Scholar 

  8. Eksombatchai, C., et al.: Pixie: a system for recommending 3+ billion items to 200+ million users in real-time. In: Proceedings of the 2018 World Wide Web Conference, WWW 2018, pp. 1775–1784. International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE (2018). https://doi.org/10.1145/3178876.3186183. ISBN 9781450356398

  9. Ferrari Dacrema, M., Boglio, S., Cremonesi, P., Jannach, D.: A troubling analysis of reproducibility and progress in recommender systems research. ACM Trans. Inf. Syst. 39(2) (2021). https://doi.org/10.1145/3434185. ISSN 1046–8188

  10. Ferrari Dacrema, M., Cremonesi, P., Jannach, D.: Are we really making much progress? A worrying analysis of recent neural recommendation approaches. In: Proceedings of the 13th ACM Conference on Recommender System, RecSys 2019, pp. 101–109. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3298689.3347058. ISBN 9781450362436

  11. Garcin, F., Faltings, B., Donatsch, O., Alazzawi, A., Bruttin, C., Huber, A.: Offline and online evaluation of news recommender systems at Swissinfo.ch. In: Proceedings of the 8th ACM Conference on Recommender Systems, RecSys 2014, pp. 169–176. Association for Computing Machinery, New York (2014). https://doi.org/10.1145/2645710.2645745. ISBN 9781450326681

  12. Gruson, A., et al.: Offline evaluation to make decisions about playlist recommendation algorithms. In: Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining, WSDM 2019, pp. 420–428. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3289600.3291027. ISBN 9781450359405

  13. Gunawardana, A., Shani, G., Yogev, S.: Evaluating recommender systems. In: Ricci, F., Rokach, L., Shapira, B. (eds.) Recommender Systems Handbook, pp. 547–601. Springer, New York (2022). https://doi.org/10.1007/978-1-0716-2197-4_15 ISBN 978-1-0716-2197-4

    Chapter  Google Scholar 

  14. Järvelin, K., Kekäläinen, J.: cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20(4), 422–446 (2002). https://doi.org/10.1145/582415.582418. ISSN 1046–8188

    Article  Google Scholar 

  15. Kersbergen, B., Sprangers, O., Schelter, S.: Serenade - low-latency session-based recommendation in e-commerce at scale. In: Proceedings of the 2022 International Conference on Management of Data, SIGMOD 2022, pp. 150–159. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3514221.3517901. ISBN 9781450392495

  16. Ludewig, M., Mauro, N., Latifi, S., Jannach, D.: Performance comparison of neural and non-neural approaches to session-based recommendation. In: Proceedings of the 13th ACM Conference on Recommender Systems, RecSys 2019, pp. 462–466. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3298689.3347041. ISBN 9781450362436

  17. Mansoury, M., Abdollahpouri, H., Pechenizkiy, M., Mobasher, B., Burke, R.: Feedback loop and bias amplification in recommender systems. In: Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM 2020, pp. 2145–2148. Association for Computing Machinery, New York (2020). https://doi.org/10.1145/3340531.3412152. ISBN 9781450368599

  18. Mei, M.J., Zuber, C., Khazaeni, Y.: A lightweight transformer for next-item product recommendation. In: Proceedings of the 16th ACM Conference on Recommender Systems, RecSys 2022, pp. 546–549. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3523227.3547491. . ISBN 9781450392785

  19. Pellegrini, R., Zhao, W., Murray, I.: Don’t recommend the obvious: estimate probability ratios. In: Proceedings of the 16th ACM Conference on Recommender Systems, RecSys 2022, pp. 188–197. Association for Computing Machinery, New York (2022). https://doi.org/10.1145/3523227.3546753. ISBN 9781450392785

  20. Rehorek, T., Biza, O., Bartyzal, R., Kordik, P., Povalyev, I., Podstavek, O.: Comparing offline and online evaluation results of recommender systems. In: REVEAL 2018: Proceedings of the Workshop on Offline Evaluation for Recommender Systems (2018). https://users.fit.cvut.cz/rehorto2/files/comparing-offline-online.pdf

  21. Rossetti, M., Stella, F., Zanker, M.: Contrasting offline and online results when evaluating recommendation algorithms. In: Proceedings of the 10th ACM Conference on Recommender Systems, RecSys 2016, pp. 31–34. Association for Computing Machinery, New York (2016). https://doi.org/10.1145/2959100.2959176. ISBN 9781450340359

  22. Verachtert, R., Michiels, L., Goethals, B.: Are we forgetting something? Correctly evaluate a recommender system with an optimal training window. In: Proceedings of the Perspectives on the Evaluation of Recommender Systems Workshop 2022, Seattle, WA, USA. CEUR-WS.org, September 2022

    Google Scholar 

  23. Zangerle, E., Bauer, C.: Evaluating recommender systems: survey and framework. ACM Comput. Surv. 55(8) (2022). https://doi.org/10.1145/3556536. ISSN 0360–0300

  24. Zhu, Z., He, Y., Zhao, X., Caverlee, J.: Popularity bias in dynamic recommendation. In: Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD 2021, pp. 2439–2449. Association for Computing Machinery, New York (2021). https://doi.org/10.1145/3447548.3467376. ISBN 9781450383325

Download references

Author information

Authors and Affiliations

  1. Froomle NV, Antwerp, Belgium

    Robin Verachtert, Jeroen Craps, Lien Michiels & Bart Goethals

  2. University of Antwerp, Antwerp, Belgium

    Robin Verachtert, Lien Michiels & Bart Goethals

  3. Monash University, Melbourne, Australia

    Bart Goethals

Authors
  1. Robin Verachtert
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeroen Craps
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lien Michiels
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Bart Goethals
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Robin Verachtert .

Editor information

Editors and Affiliations

  1. University of Amsterdam, Amsterdam, The Netherlands

    Jaap Kamps

  2. Université Grenoble-Alpes, Saint-Martin-d’Hères, France

    Lorraine Goeuriot

  3. Università della Svizzera Italiana, Lugano, Switzerland

    Fabio Crestani

  4. University of Copenhagen, Copenhagen, Denmark

    Maria Maistro

  5. University of Tsukuba, Ibaraki, Japan

    Hideo Joho

  6. Dublin City University, Dublin, Ireland

    Brian Davis

  7. Dublin City University, Dublin, Ireland

    Cathal Gurrin

  8. Universität Regensburg, Regensburg, Germany

    Udo Kruschwitz

  9. Dublin City University, Dublin, Ireland

    Annalina Caputo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Verachtert, R., Craps, J., Michiels, L., Goethals, B. (2023). The Impact of a Popularity Punishing Hyperparameter on ItemKNN Recommendation Performance. In: Kamps, J., et al. Advances in Information Retrieval. ECIR 2023. Lecture Notes in Computer Science, vol 13981. Springer, Cham. https://doi.org/10.1007/978-3-031-28238-6_56

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-28238-6_56

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-28237-9

  • Online ISBN: 978-3-031-28238-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation