Abstract
While recommender systems have mainly focused on the effectiveness of their results, beyond-accuracy perspectives have been recently explored. One of the most prominent is algorithmic bias, which analyzes if existing imbalances in the input data are exacerbated in the produced recommendations. On the other hand, calibrated recommendations ensure that the recommendations reflect the distribution of the original preferences of each user (e.g., in terms of item genres). In this paper, we connect these two perspectives, to analyze how the original calibration method deals with the bias in the state-of-the-art recommendation models. Our analysis on real-world data shows that the calibration effectiveness is impacted by how a recommendation model handles bias.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Abdollahpouri, H., Mansoury, M., Burke, R., Mobasher, B.: The unfairness of popularity bias in recommendation. arXiv preprint arXiv:1907.13286 (2019)
Anelli, V.W., et al.: Elliot: a comprehensive and rigorous framework for reproducible recommender systems evaluation, pp. 2405–2414. Association for Computing Machinery, New York (2021)
Ariza, A., Fabbri, F., Boratto, L., Salamó, M.: From the beatles to billie eilish: connecting provider representativeness and exposure in session-based recommender systems. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12657, pp. 201–208. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72240-1_16
Bella, A., Ferri, C., Hernández-Orallo, J., Ramírez-Quintana, M.J.: Calibration of machine learning models. In: Handbook of Research on Machine Learning Applications and Trends: Algorithms, Methods, and Techniques, pp. 128–146. IGI Global (2010)
Boratto, L., Fenu, G., Marras, M.: The effect of algorithmic bias on recommender systems for massive open online courses. In: Azzopardi, L., Stein, B., Fuhr, N., Mayr, P., Hauff, C., Hiemstra, D. (eds.) ECIR 2019. LNCS, vol. 11437, pp. 457–472. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-15712-8_30
Boratto, L., Fenu, G., Marras, M.: Connecting user and item perspectives in popularity debiasing for collaborative recommendation. Inf. Process. Manag. 58(1), 102387 (2021). https://doi.org/10.1016/j.ipm.2020.102387
Boratto, L., Fenu, G., Marras, M.: Interplay between upsampling and regularization for provider fairness in recommender systems. User Model. User-Adap. Inter. 31(3), 421–455 (2021). https://doi.org/10.1007/s11257-021-09294-8
Boratto, L., Fenu, G., Marras, M., Medda, G.: Consumer fairness in recommender systems: contextualizing definitions and mitigations. CoRR abs/2201.08614 (2022)
Boratto, L., Marras, M.: Hands on data and algorithmic bias in recommender systems. In: Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization, UMAP 2020, pp. 388–389. ACM (2020). https://doi.org/10.1145/3340631.3398669
Boratto, L., Marras, M.: Advances in bias-aware recommendation on the web, pp. 1147–1149. Association for Computing Machinery, New York (2021)
Boratto, L., Marras, M.: Countering bias in personalized rankings: from data engineering to algorithm development. In: 37th IEEE International Conference on Data Engineering, ICDE 2021, pp. 2362–2364. IEEE (2021). https://doi.org/10.1109/ICDE51399.2021.00266
Chen, J., Dong, H., Wang, X., Feng, F., Wang, M., He, X.: Bias and debias in recommender system: a survey and future directions (2021)
Gómez, E., Boratto, L., Salamó, M.: Disparate impact in item recommendation: a case of geographic imbalance. In: Hiemstra, D., Moens, M.-F., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds.) ECIR 2021. LNCS, vol. 12656, pp. 190–206. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72113-8_13
Gómez, E., Boratto, L., Salamó, M.: Provider fairness across continents in collaborative recommender systems. Inf. Process. Manag. 59(1), 102719 (2022). https://doi.org/10.1016/j.ipm.2021.102719
Gómez, E., Zhang, C.S., Boratto, L., Salamó, M., Marras, M.: The winner takes it all: geographic imbalance and provider (un)fairness in educational recommender systems. In: SIGIR 2021: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 1808–1812. ACM (2021). https://doi.org/10.1145/3404835.3463235
Gómez, E., Zhang, C.S., Boratto, L., Salamó, M., Ramos, G.: Enabling cross-continent provider fairness in educational recommender systems. Future Gener. Comput. Syst. 127, 435–447 (2022). https://doi.org/10.1016/j.future.2021.08.025
Kirdemir, B., Kready, J., Mead, E., Hussain, M.N., Agarwal, N.: Examining video recommendation bias on YouTube. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds.) BIAS 2021. CCIS, vol. 1418, pp. 106–116. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78818-6_10
Makhortykh, M., Urman, A., Ulloa, R.: Detecting race and gender bias in visual representation of AI on web search engines. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds.) BIAS 2021. CCIS, vol. 1418, pp. 36–50. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78818-6_5
Marlin, B.: Collaborative filtering: a machine learning perspective. University of Toronto Toronto (2004)
McGinty, L., Reilly, J.: On the evolution of critiquing recommenders. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 419–453. Springer, Boston (2011). https://doi.org/10.1007/978-0-387-85820-3_13
Nixon, J., Dusenberry, M.W., Zhang, L., Jerfel, G., Tran, D.: Measuring calibration in deep learning. In: CVPR Workshops, vol. 2 (2019)
Ramos, G., Boratto, L.: Reputation (in)dependence in ranking systems: demographics influence over output disparities. In: Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2020, pp. 2061–2064. ACM (2020). https://doi.org/10.1145/3397271.3401278
Ramos, G., Boratto, L., Marras, M.: Reputation equity in ranking systems. In: CIKM 2021: The 30th ACM International Conference on Information and Knowledge Management, Virtual Event, Queensland, Australia, 1–5 November 2021, pp. 3378–3382. ACM (2021). https://doi.org/10.1145/3459637.3482171
Seymen, S., Abdollahpouri, H., Malthouse, E.C.: A constrained optimization approach for calibrated recommendations. In: Fifteenth ACM Conference on Recommender Systems, pp. 607–612 (2021)
Steck, H.: Calibrated recommendations. In: Proceedings of the 12th ACM Conference on Recommender Systems, RecSys 2018, pp. 154–162. Association for Computing Machinery, New York (2018). https://doi.org/10.1145/3240323.3240372
Zadrozny, B., Elkan, C.: Obtaining calibrated probability estimates from decision trees and Naive Bayesian classifiers. In: ICML, vol. 1, pp. 609–616. Citeseer (2001)
Acknowledgements
M. Salamó research was partially funded by project NanoMoocs (grant No. COMRDI18-1-0010) from ACCIÓ.
D. Contreras research was partially funded by postdoctoral project (grant No. 74200094) from ANID-Chile.
Powered@NLHPC: This research was partially supported by the supercomputing infrastructure of the NLHPC (ECM-02).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Rojas, C., Contreras, D., Salamó, M. (2022). Analysis of Biases in Calibrated Recommendations. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds) Advances in Bias and Fairness in Information Retrieval. BIAS 2022. Communications in Computer and Information Science, vol 1610. Springer, Cham. https://doi.org/10.1007/978-3-031-09316-6_9
Download citation
DOI: https://doi.org/10.1007/978-3-031-09316-6_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-09315-9
Online ISBN: 978-3-031-09316-6
eBook Packages: Computer ScienceComputer Science (R0)