Wei Xia
ABSTRACT
Integrated ranking is critical in industrial recommendation systems and has attracted increasing attention. In an integrated ranking system, items from multiple channels are merged together and form an integrated list. During this process, apart from optimizing the system's utility like the total number of clicks, a fair allocation of the exposure opportunities over different channels also needs to be satisfied. To address this problem, we propose an integrated ranking model called <u>I</u>ntegrated <u>D</u>eep-<u>Q</u> <u>N</u>etwork (iDQN), which jointly considers user preferences, the platform's utility, and the exposure fairness. Extensive offline experiments validate the effectiveness of iDQN in managing the tradeoff between utility and fairness. Moreover, iDQN also has been deployed onto the online AppStore platform in Huawei, where the online A/B test shows iDQN outperforms the baseline by 1.87% and 2.21% in terms of utility and fairness, respectively.
- M. Mehdi Afsar, Trafford Crump, and Behrouz Far. 2022. Reinforcement Learning Based Recommender Systems: A Survey. ACM Comput. Surv. (jun 2022).Google Scholar
Digital Library
- Jaime Carbonell and Jade Goldstein. 1998. The use of MMR, diversity-based reranking for reordering documents and producing summaries. In Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 335--336.Google ScholarDigital Library
- Mingsheng Fu, Anubha Agrawal, Athirai A Irissappane, Jie Zhang, Liwei Huang, and Hong Qu. 2021. Deep Reinforcement Learning Framework for Category-Based Item Recommendation. IEEE Transactions on Cybernetics (2021).Google ScholarCross Ref
- Sahin Cem Geyik, Stuart Ambler, and Krishnaram Kenthapadi. 2019. Fairness-aware ranking in search & recommendation systems with application to linkedin talent search. In Proceedings of the 25th acm SIGKDD. 2221--2231.Google ScholarDigital Library
- Eric Jang, Shixiang Gu, and Ben Poole. 2016. Categorical reparameterization with gumbel-softmax. arXiv preprint arXiv:1611.01144 (2016).Google Scholar
- Diederik P. Kingma and Jimmy Ba. 2014. Adam: A Method for Stochastic Optimization. https://doi.org/10.48550/ARXIV.1412.6980Google Scholar
- Guogang Liao, Ze Wang, Xiaoxu Wu, Xiaowen Shi, Chuheng Zhang, Yongkang Wang, Xingxing Wang, and Dong Wang. 2022. Cross DQN: Cross Deep Q Network for Ads Allocation in Feed. In Proceedings of the ACM Web Conference 2022 (WWW '22). New York, NY, USA, 401--409.Google ScholarDigital Library
- Weiwen Liu, Feng Liu, Ruiming Tang, Ben Liao, Guangyong Chen, and Pheng Ann Heng. 2021. Balancing Accuracy and Fairness for Interactive Recommendation with Reinforcement Learning. arXiv preprint arXiv:2106.13386 (2021).Google Scholar
- Volodymyr Mnih, Koray Kavukcuoglu, David Silver, et al. 2015. Human-level control through deep reinforcement learning. Nat., Vol. 518, 7540 (2015), 529--533.Google ScholarCross Ref
- Ruobing Xie, Shaoliang Zhang, Rui Wang, Feng Xia, and Leyu Lin. 2021. Hierarchical Reinforcement Learning for Integrated Recommendation. Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 35, 5 (May 2021), 4521--4528.Google ScholarCross Ref
- Jinyun Yan, Zhiyuan Xu, Birjodh Tiwana, and Shaunak Chatterjee. 2020. Ads allocation in feed via constrained optimization. In Proceedings of the 26th ACM SIGKDD. 3386--3394.Google ScholarDigital Library
- Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sara Hajian, Mohamed Megahed, and Ricardo Baeza-Yates. 2017. Fa* ir: A fair top-k ranking algorithm. In Proceedings of the 2017 ACM on CIKM. 1569--1578.Google ScholarDigital Library
Index Terms
- Balancing Utility and Exposure Fairness for Integrated Ranking with Reinforcement Learning
Recommendations
Toward Pareto Efficient Fairness-Utility Trade-off in Recommendation through Reinforcement Learning
WSDM '22: Proceedings of the Fifteenth ACM International Conference on Web Search and Data MiningThe issue of fairness in recommendation is becoming increasingly essential as Recommender Systems (RS) touch and influence more and more people in their daily lives. In fairness-aware recommendation, most of the existing algorithmic approaches mainly ...
RLMixer: A Reinforcement Learning Approach for Integrated Ranking with Contrastive User Preference Modeling
Advances in Knowledge Discovery and Data MiningAbstractThere is a strong need for industrial recommender systems to output an integrated ranking of items from different categories, such as video and news, to maximize overall user satisfaction. Integrated ranking faces two critical challenges. First, ...
Integrated Ranking for News Feed with Reinforcement Learning
WWW '23 Companion: Companion Proceedings of the ACM Web Conference 2023With the development of recommender systems, it becomes an increasingly common need to mix multiple item sequences from different sources. Therefore, the integrated ranking stage is proposed to be responsible for this task with re-ranking models. ...
Comments