research-article

Public Access

Rating Distribution Calibration for Selection Bias Mitigation in Recommendations

Authors:

Youlong ChengAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2048 - 2057

https://doi.org/10.1145/3485447.3512078

Published: 25 April 2022 Publication History

All formats PDF

Abstract

Real-world recommendation datasets have been shown to be subject to selection bias, which can challenge recommendation models to learn real preferences of users, so as to make accurate recommendations. Existing approaches to mitigate selection bias, such as data imputation and inverse propensity score, are sensitive to the quality of the additional imputation or propensity estimation models. To break these limitations, in this work, we propose a novel self-supervised learning (SSL) framework, i.e., Rating Distribution Calibration (RDC), to tackle selection bias without introducing additional models. In addition to the original training objective, we introduce a rating distribution calibration loss. It aims to correct the predicted rating distribution of biased users by taking advantage of that of their similar unbiased users. We empirically evaluate RDC on two real-world datasets and one synthetic dataset. The experimental results show that RDC outperforms the original model as well as the state-of-the-art debiasing approaches by a significant margin.

References

[1]

Sanjeev Arora, Hrishikesh Khandeparkar, Mikhail Khodak, Orestis Plevrakis, and Nikunj Saunshi. 2019. A theoretical analysis of contrastive unsupervised representation learning. In 36th International Conference on Machine Learning, ICML 2019. International Machine Learning Society (IMLS), 9904–9923.

[2]

YM Asano, C Rupprecht, and A Vedaldi. 2019. A critical analysis of self-supervision, or what we can learn from a single image. In International Conference on Learning Representations.

[3]

Simon Caton and Christian Haas. 2020. Fairness in machine learning: A survey. arXiv preprint arXiv:2010.04053(2020).

[4]

Krishna Chaitanya, Ertunc Erdil, Neerav Karani, and Ender Konukoglu. 2020. Contrastive learning of global and local features for medical image segmentation with limited annotations. Advances in Neural Information Processing Systems 33 (2020).

[5]

Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2020. Bias and Debias in Recommender System: A Survey and Future Directions. arXiv preprint arXiv:2010.03240(2020).

[6]

Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PMLR, 1597–1607.

[7]

Carl Doersch, Abhinav Gupta, and Alexei A Efros. 2015. Unsupervised visual representation learning by context prediction. In Proceedings of the IEEE international conference on computer vision. 1422–1430.

Digital Library

[8]

Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728(2018).

[9]

F Maxwell Harper and Joseph A Konstan. 2015. The movielens datasets: History and context. Acm transactions on interactive intelligent systems (tiis) 5, 4(2015), 1–19.

[10]

Xiangnan He and Tat-Seng Chua. 2017. Neural factorization machines for sparse predictive analytics. In Proceedings of the 40th International ACM SIGIR conference on Research and Development in Information Retrieval. 355–364.

Digital Library

[11]

José Miguel Hernández-Lobato, Neil Houlsby, and Zoubin Ghahramani. 2014. Probabilistic matrix factorization with non-random missing data. In International Conference on Machine Learning. PMLR, 1512–1520.

[12]

Ashish Jaiswal, Ashwin Ramesh Babu, Mohammad Zaki Zadeh, Debapriya Banerjee, and Fillia Makedon. 2021. A survey on contrastive self-supervised learning. Technologies 9, 1 (2021), 2.

[13]

Wei Jin, Tyler Derr, Haochen Liu, Yiqi Wang, Suhang Wang, Zitao Liu, and Jiliang Tang. 2020. Self-supervised learning on graphs: Deep insights and new direction. arXiv preprint arXiv:2006.10141(2020).

[14]

Phuc H Le-Khac, Graham Healy, and Alan F Smeaton. 2020. Contrastive representation learning: A framework and review. IEEE Access (2020).

[15]

Haochen Liu, Yiqi Wang, Wenqi Fan, Xiaorui Liu, Yaxin Li, Shaili Jain, Yunhao Liu, Anil K Jain, and Jiliang Tang. 2021. Trustworthy ai: A computational perspective. arXiv preprint arXiv:2107.06641(2021).

[16]

Zhuang Liu, Yunpu Ma, Yuanxin Ouyang, and Zhang Xiong. 2021. Contrastive Learning for Recommender System. arXiv preprint arXiv:2101.01317(2021).

[17]

Benjamin M Marlin and Richard S Zemel. 2009. Collaborative prediction and ranking with non-random missing data. In Proceedings of the third ACM conference on Recommender systems. 5–12.

Digital Library

[18]

Benjamin M Marlin, Richard S Zemel, Sam Roweis, and Malcolm Slaney. 2007. Collaborative filtering and the missing at random assumption. In Proceedings of the Twenty-Third Conference on Uncertainty in Artificial Intelligence. 267–275.

Digital Library

[19]

Alejandro Newell and Jia Deng. 2020. How useful is self-supervised pretraining for visual tasks?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 7345–7354.

[20]

Mehdi Noroozi and Paolo Favaro. 2016. Unsupervised learning of visual representations by solving jigsaw puzzles. In European conference on computer vision. Springer, 69–84.

[21]

Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. 2017. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning. PMLR, 2778–2787.

[22]

Deepak Pathak, Ross Girshick, Piotr Dollár, Trevor Darrell, and Bharath Hariharan. 2017. Learning features by watching objects move. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2701–2710.

[23]

Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. 2016. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2536–2544.

[24]

Francesco Ricci, Lior Rokach, and Bracha Shapira. 2011. Introduction to recommender systems handbook. In Recommender systems handbook. Springer, 1–35.

[25]

Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as treatments: Debiasing learning and evaluation. In international conference on machine learning. PMLR, 1670–1679.

[26]

Harald Steck. 2010. Training and testing of recommender systems on data missing not at random. In Proceedings of the 16th ACM SIGKDD international conference on Knowledge discovery and data mining. 713–722.

Digital Library

[27]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly robust joint learning for recommendation on data missing not at random. In International Conference on Machine Learning. PMLR, 6638–6647.

[28]

Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2021. Combating Selection Biases in Recommender Systems with a Few Unbiased Ratings. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 427–435.

Digital Library

[29]

Jiawei Wu, Xin Wang, and William Yang Wang. 2019. Self-Supervised Dialogue Learning. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. 3857–3867.

[30]

Yuning You, Tianlong Chen, Zhangyang Wang, and Yang Shen. 2020. When does self-supervision help graph convolutional networks?. In International Conference on Machine Learning. PMLR, 10871–10880.

[31]

Kun Zhou, Hui Wang, Wayne Xin Zhao, Yutao Zhu, Sirui Wang, Fuzheng Zhang, Zhongyuan Wang, and Ji-Rong Wen. 2020. S3-rec: Self-supervised learning for sequential recommendation with mutual information maximization. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 1893–1902.

Digital Library

Cited By

Yang XLi XLiu ZYuan YWang Y(2025)Multi-teacher knowledge distillation for debiasing recommendation with uniform dataExpert Systems with Applications10.1016/j.eswa.2025.126808273(126808)Online publication date: May-2025
https://doi.org/10.1016/j.eswa.2025.126808
Wang Z(2024)Enhancing User-Item Interaction Through Counterfactual Classifier For Sequential RecommendationApplied Mathematics and Nonlinear Sciences10.2478/amns-2024-24819:1Online publication date: 3-Sep-2024
https://doi.org/10.2478/amns-2024-2481
Cavenaghi EZanga AStella FZanker M(2024)Towards a Causal Decision-Making Framework for Recommender SystemsACM Transactions on Recommender Systems10.1145/36291692:2(1-34)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3629169
Show More Cited By

Index Terms

Rating Distribution Calibration for Selection Bias Mitigation in Recommendations
1. Computing methodologies
  1. Machine learning
2. Information systems
  1. Information retrieval
    1. Retrieval tasks and goals
  2. Information systems applications
    1. Data mining

Index terms have been assigned to the content through auto-classification.

Recommendations

Generating Items Recommendations by Fusing Content and User-Item based Collaborative Filtering
Abstract
Nowadays e-commerce has spread all over the world. The e-shops are not similar to the physical shops. The e-shops can have hundreds or thousands of items independent of physical boundaries. The information about all these products is available on ...
Improvement of collaborative filtering using rating normalization

With the advent of the Internet, the types and amount of information one can access have increased dramatically. In today's overwhelming information environment, recommendation systems that quickly analyze large amounts of available information and help ...
Generating Top-N Items Recommendation Set Using Collaborative, Content Based Filtering and Rating Variance
Abstract
The main purpose of any recommendation system is to recommend items of users’ interest. Mostly content and collaborative filtering are widely used recommendation systems. Matrix factorization technique is also used by many recommendation systems. ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
2,148
Total Downloads

Downloads (Last 12 months)517
Downloads (Last 6 weeks)56

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang XLi XLiu ZYuan YWang Y(2025)Multi-teacher knowledge distillation for debiasing recommendation with uniform dataExpert Systems with Applications10.1016/j.eswa.2025.126808273(126808)Online publication date: May-2025
https://doi.org/10.1016/j.eswa.2025.126808
Wang Z(2024)Enhancing User-Item Interaction Through Counterfactual Classifier For Sequential RecommendationApplied Mathematics and Nonlinear Sciences10.2478/amns-2024-24819:1Online publication date: 3-Sep-2024
https://doi.org/10.2478/amns-2024-2481
Cavenaghi EZanga AStella FZanker M(2024)Towards a Causal Decision-Making Framework for Recommender SystemsACM Transactions on Recommender Systems10.1145/36291692:2(1-34)Online publication date: 14-May-2024
https://dl.acm.org/doi/10.1145/3629169
Su HMeng LZhu LLu KLi JHui Yang GWang HHan SHauff CZuccon GZhang Y(2024)DDPO: Direct Dual Propensity Optimization for Post-Click Conversion Rate EstimationProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval10.1145/3626772.3657817(1179-1188)Online publication date: 10-Jul-2024
https://dl.acm.org/doi/10.1145/3626772.3657817
Chen QWei BCheng DLi JLiu LZhang S(2024)A Novel Shadow Variable Catcher for Addressing Selection Bias in Recommendation Systems2024 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM59182.2024.00014(71-80)Online publication date: 9-Dec-2024
https://doi.org/10.1109/ICDM59182.2024.00014
Zhang HWang SLi HZheng CChen XLiu LLuo SWu P(2024)Uncovering the Propensity Identification Problem in Debiased Recommendations2024 IEEE 40th International Conference on Data Engineering (ICDE)10.1109/ICDE60146.2024.00056(653-666)Online publication date: 13-May-2024
https://doi.org/10.1109/ICDE60146.2024.00056
Zhang QZhang XLiu YWang HGao MZhang JGuo RSingh ASun YAkoglu LGunopulos DYan XKumar ROzcan FYe J(2023)Debiasing Recommendation by Learning Identifiable Latent ConfoundersProceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining10.1145/3580305.3599296(3353-3363)Online publication date: 6-Aug-2023
https://dl.acm.org/doi/10.1145/3580305.3599296
Wang JLi HZhang CLiang DYu EOu WWang W(2023)CounterCLR: Counterfactual Contrastive Learning with Non-random Missing Data in Recommendation2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00174(1355-1360)Online publication date: 1-Dec-2023
https://doi.org/10.1109/ICDM58522.2023.00174
Jin DWang LZhang HZheng YDing WXia FPan S(2023)A survey on fairness-aware recommender systemsInformation Fusion10.1016/j.inffus.2023.101906100:COnline publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1016/j.inffus.2023.101906

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten