skip to main content
10.1145/3580305.3599550acmconferencesArticle/Chapter ViewAbstractPublication PageskddConference Proceedingsconference-collections
research-article

Who Should Be Given Incentives? Counterfactual Optimal Treatment Regimes Learning for Recommendation

Published: 04 August 2023 Publication History

Abstract

Effective personalized incentives can improve user experience and increase platform revenue, resulting in a win-win situation between users and e-commerce companies. Previous studies have used uplift modeling methods to estimate the conditional average treatment effects of users' incentives, and then placed the incentives by maximizing the sum of estimated treatment effects under a limited budget. However, some users will always buy whether incentives are given or not, and they will actively collect and use incentives if provided, named "Always Buyers". Identifying and predicting these "Always Buyers" and reducing incentive delivery to them can lead to a more rational incentive allocation. In this paper, we first divide users into five strata from an individual counterfactual perspective, and reveal the failure of previous uplift modeling methods to identify and predict the "Always Buyers". Then, we propose principled counterfactual identification and estimation methods and prove their unbiasedness. We further propose a counterfactual entire-space multi-task learning approach to accurately perform personalized incentive policy learning with a limited budget. We also theoretically derive a lower bound on the reward of the learned policy. Extensive experiments are conducted on three real-world datasets with two common incentive scenarios, and the results demonstrate the effectiveness of the proposed approaches.

References

[1]
Qingyao Ai, Keping Bi, Cheng Luo, Jiafeng Guo, and W Bruce Croft. 2018. Unbiased learning to rank with unbiased propensity estimation. In SIGIR.
[2]
Nabiha Asghar. 2016. Yelp dataset challenge: Review rating prediction. arXiv preprint arXiv:1605.05362 (2016).
[3]
Stephen Bonner and Flavian Vasile. 2018. Causal embeddings for recommendation. In Proceedings of the 12th ACM Conference on Recommender Systems. 104--112.
[4]
Chih-Yao Chang, Xing Tang, Bo-Wen Yuan, Jui-Yang Hsia, Zhirong Liu, Zhenhua Dong, Xiuqiang He, and Chih-Jen Lin. 2020. AutoConjunction: Adaptive Model-based Feature Conjunction for CTR Prediction. In MDM. IEEE, 202--209.
[5]
Jiawei Chen, Hande Dong, Yang Qiu, Xiangnan He, Xin Xin, Liang Chen, Guli Lin, and Keping Yang. 2021. AutoDebias: Learning to Debias for Recommendation. In SIGIR.
[6]
Jiawei Chen, Hande Dong, Xiang Wang, Fuli Feng, Meng Wang, and Xiangnan He. 2023. Bias and debias in recommender system: A survey and future directions. ACM Transactions on Information Systems, Vol. 41, 3 (2023), 1--39.
[7]
Hugh A Chipman, Edward I George, and Robert E McCulloch. 2010. BART: Bayesian additive regression trees. The Annals of Applied Statistics, Vol. 4, 1 (2010), 266--298.
[8]
Quanyu Dai, Haoxuan Li, Peng Wu, Zhenhua Dong, Xiao-Hua Zhou, Rui Zhang, Xiuqiang He, Rui Zhang, and Jie Sun. 2022. A Generalized Doubly Robust Learning Framework for Debiasing Post-Click Conversion Rate Prediction. In KDD.
[9]
Zhenhua Dong, Hong Zhu, Pengxiang Cheng, Xinhua Feng, Guohao Cai, Xiuqiang He, Jun Xu, and Jirong Wen. 2020. Counterfactual learning for recommender system. In Fourteenth ACM Conference on Recommender Systems. 568--569.
[10]
Chongming Gao, Wenqiang Lei, Jiawei Chen, Shiqi Wang, Xiangnan He, Shijun Li, Biao Li, Yuan Zhang, and Peng Jiang. 2022a. Cirs: Bursting filter bubbles by counterfactual interactive recommender system. arXiv preprint arXiv:2204.01266 (2022).
[11]
Chongming Gao, Shijun Li, Wenqiang Lei, Jiawei Chen, Biao Li, Peng Jiang, Xiangnan He, Jiaxin Mao, and Tat-Seng Chua. 2022b. KuaiRec: A Fully-observed Dataset and Insights for Evaluating Recommender Systems. In CIKM (Atlanta, GA, USA). 11 pages.
[12]
Brett R Gordon, Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky. 2019. A comparison of approaches to advertising measurement: Evidence from big field experiments at Facebook. Marketing Science, Vol. 38, 2 (2019), 193--225.
[13]
Tiankai Gu, Kun Kuang, Hong Zhu, Jingjie Li, Zhenhua Dong, Wenjie Hu, Zhenguo Li, Xiuqiang He, and Yue Liu. 2021. Estimating True Post-Click Conversion via Group-stratified Counterfactual Inference. In ADKDD.
[14]
Leo Guelman, Montserrat Guillén, and Ana M Pérez-Marín. 2015. A decision support framework to implement optimal personalized marketing interventions. Decision Support Systems, Vol. 72 (2015), 24--32.
[15]
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, Xiuqiang He, and Zhenhua Dong. 2018. Deepfm: An end-to-end wide & deep learning framework for CTR prediction. arXiv preprint arXiv:1804.04950 (2018).
[16]
Siyuan Guo, Lixin Zou, Yiding Liu, Wenwen Ye, Suqi Cheng, Shuaiqiang Wang, Hechang Chen, Dawei Yin, and Yi Chang. 2021. Enhanced Doubly Robust Learning for Debiasing Post-Click Conversion Rate Estimation. In SIGIR.
[17]
Pierre Gutierrez and Jean-Yves Gérardy. 2017. Causal inference and uplift modelling: A review of the literature. In Predictive Applications and APIs. 1--13.
[18]
Behram Hansotia and Brad Rukstales. 2002. Incremental value modeling. Journal of Interactive Marketing, Vol. 16 (2002), 35--46.
[19]
Xiangnan He, Lizi Liao, Hanwang Zhang, Liqiang Nie, Xia Hu, and Tat-Seng Chua. 2017. Neural collaborative filtering. In WWW.
[20]
M.A. Hernán and J. M. Robins. 2020. Causal Inference: What If. Boca Raton: Chapman and Hall/CRC.
[21]
Paul W Holland. 1986. Statistics and causal inference. Journal of the American statistical Association (1986), 945--960.
[22]
D. G. Horvitz and D. J. Thompson. 1952. A Generalization of Sampling Without Replacement From a Finite Universe. Journal of the American Statistical Association, Vol. 47 (1952), 663--685.
[23]
G. W. Imbens and D. B. Rubin. 2015a. Causal Inference For Statistics Social and Biomedical Science. Cambridge University Press.
[24]
Guido W Imbens and Donald B Rubin. 2015b. Causal inference in statistics, social, and biomedical sciences. Cambridge University Press.
[25]
Maciej Jaskowski and Szymon Jaroszewicz. 2012. Uplift modeling for clinical trial data. In ICML Workshop on Clinical Data Analysis, Vol. 46. 79--95.
[26]
Fredrik Johansson, Uri Shalit, and David Sontag. 2016. Learning representations for counterfactual inference. In International conference on machine learning. PMLR, 3020--3029.
[27]
Edward H. Kennedy. 2020. Optimal doubly robust estimation of heterogeneous causal effects. https://arxiv.org/abs/2004.14497v1 (2020).
[28]
Sami Khenissi and Olfa Nasraoui. 2020. Modeling and Counteracting Exposure Bias in Recommender Systems. arXiv:2001.04832 (2020).
[29]
Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. 2019. Metalearners for estimating heterogeneous treatment effects using machine learning. Proceedings of the national academy of sciences, Vol. 116, 10 (2019), 4156--4165.
[30]
Haoxuan Li, Quanyu Dai, Yuru Li, Yan Lyu, Zhenhua Dong, Xiao-Hua Zhou, and Peng Wu. 2023 a. Multiple Robust Learning for Recommendation. In AAAI.
[31]
Haoxuan Li, Yan Lyu, Chunyuan Zheng, and Peng Wu. 2023 b. TDR-CL: Targeted Doubly Robust Collaborative Learning for Debiased Recommendations. In ICLR.
[32]
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, and Peng Wu. 2023 c. Balancing Unobserved Confounding with a Few Unbiased Ratings in Debiased Recommendations. In WWW.
[33]
Haoxuan Li, Yanghao Xiao, Chunyuan Zheng, Peng Wu, and Peng Cui. 2023 d. Propensity Matters: Measuring and Enhancing Balancing for Recommendation. In ICML.
[34]
Haoxuan Li, Chunyuan Zheng, Yixiao Cao, Zhi Geng, Yue Liu, and Peng Wu. 2023 f. Trustworthy Policy Learning under the Counterfactual No-Harm Criterion. In ICML.
[35]
Haoxuan Li, Chunyuan Zheng, and Peng Wu. 2023 e. StableDR: Stabilized Doubly Robust Learning for Recommendation on Data Missing Not at Random. In ICLR.
[36]
Dugang Liu, Pengxiang Cheng, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2020a. A general knowledge distillation framework for counterfactual recommendation via uniform data. In SIGIR. 831--840.
[37]
Dugang Liu, Pengxiang Cheng, Hong Zhu, Zhenhua Dong, Xiuqiang He, Weike Pan, and Zhong Ming. 2021. Mitigating Confounding Bias in Recommendation via Information Bottleneck. In RecSys.
[38]
Huafeng Liu, Liping Jing, Jingxuan Wen, Zhicheng Wu, Xiaoyi Sun, Jiaqi Wang, Lin Xiao, and Jian Yu. 2020b. Deep global and local generative model for recommendation. In Proceedings of The Web Conference 2020. 551--561.
[39]
Huafeng Liu, Jingxuan Wen, Liping Jing, and Jian Yu. 2019. Deep generative ranking for personalized recommendation. In Proceedings of the 13th ACM Conference on Recommender Systems. 34--42.
[40]
Yiming Liu, Xuezhi Cao, and Yong Yu. 2016. Are You Influenced by Others When Rating?: Improve Rating Prediction by Conformity Modeling. In RecSys. 269--272.
[41]
Jiaqi Ma, Zhe Zhao, Xinyang Yi, Jilin Chen, Lichan Hong, and Ed H Chi. 2018b. Modeling task relationships in multi-task learning with multi-gate mixture-of-experts. In KDD. 1930--1939.
[42]
Xiao Ma, Liqin Zhao, Guan Huang, Zhi Wang, Zelin Hu, Xiaoqiang Zhu, and Kun Gai. 2018a. Entire Space Multi-Task Model: An Effective Approach for Estimating Post-Click Conversion Rate. In SIGIR.
[43]
Benjamin Marlin, Richard S Zemel, Sam Roweis, and Malcolm Slaney. 2007. Collaborative filtering and the missing at random assumption. UAI (2007).
[44]
Rishabh Mehrotra, Prasanta Bhattacharya, and Mounia Lalmas. 2020. Inferring the Causal Impact of New Track Releases on Music Recommendation Platforms through Counterfactual Predictions. In RecSys. 687--691.
[45]
Mehryar Mohri, Afshin Rostamizadeh, and Ameet Talwalkar. 2018. Foundations of Machine Learning. MIT Press.
[46]
X Nie and S Wager. 2021. Quasi-oracle estimation of heterogeneous treatment effects. Biometrika, Vol. 108 (2021), 4156--4165.
[47]
J.M. Robins, A. Rotnitzky, and L.P. Zhao. 1994. Estimation of regression coefficients when some regressors are not always observed. J. Amer. Statist. Assoc., Vol. 89 (1994), 846--866.
[48]
Donald B Rubin. 2005. Causal inference using potential outcomes: Design, modeling, decisions. J. Amer. Statist. Assoc., Vol. 100, 469 (2005), 322--331.
[49]
Yuta Saito. 2020. Doubly robust estimator for ranking metrics with post-click conversions. In Fourteenth ACM Conference on Recommender Systems. 92--100.
[50]
Yuta Saito, Hayato Sakata, and Kazuhide Nakata. 2019. Doubly Robust Prediction and Evaluation Methods Improve Uplift Modeling for Observational Data. In SIAM.
[51]
Yuta Saito, Hayato Sakata, and Kazuhide Nakata. 2020. Cost-effective and stable policy optimization algorithm for uplift modeling with multiple treatments. In Proceedings of the 2020 SIAM International Conference on Data Mining. SIAM, 406--414.
[52]
Masahiro Sato, Janmajay Singh, Sho Takemori, Takashi Sonoda, Qian Zhang, and Tomoko Ohkuma. 2019. Uplift-based evaluation and optimization of recommenders. In Proceedings of the 13th ACM Conference on Recommender Systems. 296--304.
[53]
Tobias Schnabel, Adith Swaminathan, Ashudeep Singh, Navin Chandak, and Thorsten Joachims. 2016. Recommendations as Treatments: Debiasing Learning and Evaluation. In ICML.
[54]
Patrick Schwab, Lorenz Linhardt, and Walter Karlen. 2018. Perfect match: A simple method for learning representations for counterfactual inference with neural networks. arXiv preprint arXiv:1810.00656 (2018).
[55]
Uri Shalit, Fredrik D Johansson, and David Sontag. 2017. Estimating individual treatment effect: generalization bounds and algorithms. In International Conference on Machine Learning. PMLR, 3076--3085.
[56]
Claudia Shi, David Blei, and Victor Veitch. 2019. Adapting neural networks for the estimation of treatment effects. Advances in neural information processing systems, Vol. 32 (2019).
[57]
Harald Steck. 2010. Training and testing of recommender systems on data missing not at random. In KDD.
[58]
H. Varian. 2016. Causal inference in economics and marketing. Proceedings of the National Academy of Sciences, Vol. 113 (2016), 7310--7315.
[59]
Stefan Wager and Susan Athey. 2018. Estimation and inference of heterogeneous treatment effects using random forests. J. Amer. Statist. Assoc., Vol. 113, 523 (2018), 1228--1242.
[60]
Hao Wang, Tai-Wei Chang, Tianqiao Liu, Jianmin Huang, Zhichao Chen, Chao Yu, Ruopeng Li, and Wei Chu. 2022. ESCM 2: Entire Space Counterfactual Multi-Task Model for Post-Click Conversion Rate Estimation. In SIGIR.
[61]
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017. Deep & cross network for ad click predictions. In ADKDD. 1--7.
[62]
Wenjie Wang, Fuli Feng, Xiangnan He, Hanwang Zhang, and Tat-Seng Chua. 2021a. Clicks can be cheating: Counterfactual recommendation for mitigating clickbait issue. In SIGIR. 1288--1297.
[63]
Wenjie Wang, Yang Zhang, Haoxuan Li, Peng Wu, Fuli Feng, and Xiangnan He. 2023. Causal Recommendation: Progresses and Future Directions. In Tutorial on SIGIR.
[64]
Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2019. Doubly Robust Joint Learning for Recommendation on Data Missing Not at Random. In ICML.
[65]
Xiaojie Wang, Rui Zhang, Yu Sun, and Jianzhong Qi. 2021b. Combating Selection Biases in Recommender Systems with A Few Unbiased Ratings. In WSDM.
[66]
Zifeng Wang, Xi Chen, Rui Wen, Shao-Lun Huang, Ercan Kuruoglu, and Yefeng Zheng. 2020. Information theoretic counterfactual learning from missing-not-at-random feedback. NeurIPS (2020), 1854--1864.
[67]
Tianxin Wei, Fuli Feng, Jiawei Chen, Ziwei Wu, Jinfeng Yi, and Xiangnan He. 2021. Model-agnostic counterfactual reasoning for eliminating popularity bias in recommender system. In SIGKDD. 1791--1800.
[68]
Liang Wu, Diane Hu, Liangjie Hong, and Huan Liu. 2018. Turning Clicks into Purchases: Revenue Optimization for Product Search in E-Commerce. In SIGIR.
[69]
Peng Wu, Haoxuan Li, Yuhao Deng, Wenjie Hu, Quanyu Dai, Zhenhua Dong, Jie Sun, Rui Zhang, and Xiao-Hua Zhou. 2022. On the Opportunity of Causal Learning in Recommendation Systems: Foundation, Estimation, Prediction and Challenges. In IJCAI.
[70]
Yao Wu, Christopher DuBois, Alice X Zheng, and Martin Ester. 2016a. Collaborative denoising auto-encoders for top-n recommender systems. In Proceedings of the ninth ACM international conference on web search and data mining. 153--162.
[71]
Yao Wu, Xudong Liu, Min Xie, Martin Ester, and Qing Yang. 2016b. CCCF: Improving collaborative filtering via scalable user-item co-clustering. In Proceedings of the ninth ACM international conference on web search and data mining. 73--82.
[72]
Mengyue Yang, Quanyu Dai, Zhenhua Dong, Xu Chen, Xiuqiang He, and Jun Wang. 2021. Top-N Recommendation with Counterfactual User Preference Simulation. In CIKM. 2342--2351.
[73]
Bowen Yuan, Jui-Yang Hsia, Meng-Yuan Yang, Hong Zhu, Chih-Yao Chang, Zhenhua Dong, and Chih-Jen Lin. 2019. Improving ad click prediction by considering non-displayed events. In CIKM. 329--338.
[74]
Bowen Yuan, Yaxu Liu, Jui-Yang Hsia, Zhenhua Dong, and Chih-Jen Lin. 2020. Unbiased Ad Click Prediction for Position-aware Advertising Systems. In Fourteenth ACM Conference on Recommender Systems. 368--377.
[75]
Wenhao Zhang, Wentian Bao, Xiao-Yang Liu, Keping Yang, Quan Lin, Hong Wen, and Ramin Ramezani. 2020. Large-scale Causal Approaches to Debiasing Post-click Conversion Rate Estimation with Multi-task Learning. In WWW. 2775--2781.
[76]
Yang Zhang, Fuli Feng, Xiangnan He, Tianxin Wei, Chonggang Song, Guohui Ling, and Yongdong Zhang. 2021. Causal intervention for leveraging popularity bias in recommendation. In SIGIR. 11--20.
[77]
Kailiang Zhong, Fengtong Xiao, Yan Ren, Yaorong Liang, Wenqing Yao, Xiaofeng Yang, and Ling Cen. 2022. DESCN: Deep Entire Space Cross Networks for Individual Treatment Effect Estimation. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. 4612--4620.

Cited By

View all
  • (2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
  • (2024)An Accurate and Interpretable Framework for Trustworthy Process MonitoringIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33196065:5(2241-2252)Online publication date: May-2024
  • (2024)Towards Integrated Energy-Communication-Transportation Hub: A Base-Station-Centric Design in 5G and Beyond2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00090(925-935)Online publication date: 23-Jul-2024
  • Show More Cited By

Index Terms

  1. Who Should Be Given Incentives? Counterfactual Optimal Treatment Regimes Learning for Recommendation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    KDD '23: Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining
    August 2023
    5996 pages
    ISBN:9798400701030
    DOI:10.1145/3580305
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 04 August 2023

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. counterfactual
    2. optimal treatment regime
    3. recommender system.

    Qualifiers

    • Research-article

    Conference

    KDD '23
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 1,133 of 8,635 submissions, 13%

    Upcoming Conference

    KDD '25

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)604
    • Downloads (Last 6 weeks)46
    Reflects downloads up to 17 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Semantic Codebook Learning for Dynamic Recommendation ModelsProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680574(9611-9620)Online publication date: 28-Oct-2024
    • (2024)An Accurate and Interpretable Framework for Trustworthy Process MonitoringIEEE Transactions on Artificial Intelligence10.1109/TAI.2023.33196065:5(2241-2252)Online publication date: May-2024
    • (2024)Towards Integrated Energy-Communication-Transportation Hub: A Base-Station-Centric Design in 5G and Beyond2024 IEEE 44th International Conference on Distributed Computing Systems (ICDCS)10.1109/ICDCS60910.2024.00090(925-935)Online publication date: 23-Jul-2024
    • (2023)Removing hidden confounding in recommendationProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668502(54614-54626)Online publication date: 10-Dec-2023
    • (2023)Optimal transport for treatment effect estimationProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3666359(5404-5418)Online publication date: 10-Dec-2023
    • (2023)Propensity mattersProceedings of the 40th International Conference on Machine Learning10.5555/3618408.3619239(20182-20194)Online publication date: 23-Jul-2023
    • (2023)Pareto Invariant Representation Learning for Multimedia RecommendationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612591(6410-6419)Online publication date: 26-Oct-2023
    • (2023)CounterCLR: Counterfactual Contrastive Learning with Non-random Missing Data in Recommendation2023 IEEE International Conference on Data Mining (ICDM)10.1109/ICDM58522.2023.00174(1355-1360)Online publication date: 1-Dec-2023

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media