short-paper

Optimizing Reciprocal Rank with Bayesian Average for improved Next Item Recommendation

Authors:

Jianbo YuanAuthors Info & Claims

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

Pages 2236 - 2240

https://doi.org/10.1145/3539618.3592033

Published: 18 July 2023 Publication History

Abstract

Next item recommendation is a crucial task of session-based recommendation. However, the gap between the optimization objective (Binary Cross Entropy) and the ranking metric (Mean Reciprocal Rank) has not been well-explored, resulting in sub-optimal recommendations. In this paper, we propose a novel objective function, namely Adjusted-RR, to directly optimize Mean Reciprocal Rank. Specifically, Adjusted-RR adopts Bayesian Average to adjust Reciprocal Rank loss with Normal Rank loss by creating position-aware weights between them. Adjusted-RR is a plug-and-play objective that is compatible with various models. We apply Adjusted-RR on two base models and two datasets, and experimental results show that it makes a significant improvement in the next item recommendation.

Supplemental Material

MP4 File

To address the problem of inconsistency between the optimization objective (Binary Cross Entropy, BCE) and the benchmark metric (Mean Reciprocal Rank, MRR) in the session-based recommendation, we propose a novel objective function, named Adjusted-RR, to optimize MRR metric directly by employing Bayesian average to adjust RR (Reciprocal Rank) loss with NR(Normal Rank) loss. Experimental results demonstrate consistent improvements of Adjusted-RR when incorporated into two backbones.

Download
120.74 MB

References

[1]

Suhrid Balakrishnan and Sumit Chopra. 2012. Collaborative ranking. In WSDM. ACM, 143--152. https://doi.org/10.1145/2124295.2124314

Digital Library

[2]

Andrew Brown, Weidi Xie, Vicky Kalogeiton, and Andrew Zisserman. 2020. Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval. In ECCV, Vol. 12354. 677--694. https://doi.org/10.1007/978-3-030-58545-7_39

Digital Library

[3]

Fatih Cakir, Kun He, Xide Xia, Brian Kulis, and Stan Sclaroff. 2019. Deep Metric Learning to Rank. In CVPR. 1861--1870. https://doi.org/10.1109/CVPR.2019.00196

[4]

Paul Henderson and Vittorio Ferrari. 2016. End-to-End Training of Object Class Detectors for Mean Average Precision. In ACCV, Vol. 10115. Springer, 198--213. https://doi.org/10.1007/978-3-319-54193-8_13

[5]

Alan Herschtal and Bhavani Raskutti. 2004. Optimising area under the ROC curve using gradient descent. In ICML, Vol. 69. ACM. https://doi.org/10.1145/1015330.1015366

Digital Library

[6]

Hang Li. 2011. A Short Introduction to Learning to Rank. IEICE Transactions, Vol. 94-D (10 2011), 1854--1862.

[7]

Pritish Mohapatra, Michal Rolínek, C. V. Jawahar, Vladimir Kolmogorov, and M. Pawan Kumar. 2018. Efficient Optimization for Rank-Based Loss Functions. In CVPR. Computer Vision Foundation / IEEE Computer Society, 3693--3701.

[8]

Yitong Pang, Lingfei Wu, Qi Shen, Yiming Zhang, Zhihua Wei, Fangli Xu, Ethan Chang, Bo Long, and Jian Pei. 2022. Heterogeneous Global Graph Neural Networks for Personalized Session-based Recommendation. In WSDM. ACM, 775--783. https://doi.org/10.1145/3488560.3498505

Digital Library

[9]

Michal Rolinek, Vit Musil, Anselm Paulus, Marin Vlastelica, Claudio Michaelis, and Georg Martius. 2020. Optimizing Rank-Based Metrics With Blackbox Differentiation. In CVPR.

[10]

Song-Qing Shen, Bin-Bin Yang, and Wei Gao. 2020. AUC Optimization with a Reject Option. In AAAI. AAAI Press, 5684--5691. https://ojs.aaai.org/index.php/AAAI/article/view/6023

[11]

Yue Shi, Alexandros Karatzoglou, Linas Baltrunas, Martha A. Larson, Nuria Oliver, and Alan Hanjalic. 2012. CLiMF: learning to maximize reciprocal rank with collaborative less-is-more filtering. In RecSys. ACM, 139--146. https://doi.org/10.1145/2365952.2365981

Digital Library

[12]

Yue Shi, Martha A. Larson, and Alan Hanjalic. 2010. List-wise learning to rank with matrix factorization for collaborative filtering. In RecSys. ACM, 269--272. https://doi.org/10.1145/1864708.1864764

Digital Library

[13]

Hyun Oh Song, Yu Xiang, Stefanie Jegelka, and Silvio Savarese. 2016. Deep Metric Learning via Lifted Structured Feature Embedding. In CVPR. IEEE Computer Society, 4004--4012. https://doi.org/10.1109/CVPR.2016.434

[14]

Shuang Tang, Fangyuan Luo, and Jun Wu. 2022. Smooth-AUC: Smoothing the Path Towards Rank-based CTR Prediction. In SIGIR. ACM, 2400--2404. https://doi.org/10.1145/3477495.3531865

Digital Library

[15]

Shoujin Wang, Longbing Cao, Yan Wang, Quan Z. Sheng, Mehmet A. Orgun, and Defu Lian. 2022. A Survey on Session-based Recommender Systems. ACM Comput. Surv., Vol. 54, 7 (2022), 154:1--154:38. https://doi.org/10.1145/3465401

Digital Library

[16]

Ziyang Wang, Wei Wei, Gao Cong, Xiao-Li Li, Xianling Mao, and Minghui Qiu. 2020. Global Context Enhanced Graph Neural Networks for Session-based Recommendation. In SIGIR. ACM, 169--178. https://doi.org/10.1145/3397271.3401142

Digital Library

[17]

Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019a. Session-Based Recommendation with Graph Neural Networks. In AAAI. AAAI Press, 346--353. https://doi.org/10.1609/aaai.v33i01.3301346

Digital Library

[18]

Shu Wu, Yuyuan Tang, Yanqiao Zhu, Liang Wang, Xing Xie, and Tieniu Tan. 2019b. Session-Based Recommendation with Graph Neural Networks. In AAAI. AAAI Press, 346--353. https://doi.org/10.1609/aaai.v33i01.3301346

Digital Library

[19]

Mengqi Zhang, Shu Wu, Meng Gao, Xin Jiang, Ke Xu, and Liang Wang. 2022. Personalized Graph Neural Networks With Attention Mechanism for Session-Aware Recommendation. IEEE Trans. Knowl. Data Eng., Vol. 34, 8 (2022), 3946--3957. https://doi.org/10.1109/TKDE.2020.3031329

Cited By

Lu XWu J(2025)Combining association-rule-guided sequence augmentation with listwise contrastive learning for session-based recommendationInformation Processing & Management10.1016/j.ipm.2024.10399962:3(103999)Online publication date: May-2025
https://doi.org/10.1016/j.ipm.2024.103999
Wei LLi YWang WZhu Y(2024)Enhancing Knowledge-Concept Recommendations with Heterogeneous Graph-Contrastive LearningMathematics10.3390/math1215232412:15(2324)Online publication date: 25-Jul-2024
https://doi.org/10.3390/math12152324
Li YZhang HJin YRen ZDong LLyu JYang LLo DShao DFilkov VRay BZhou M(2024)An Explainable Automated Model for Measuring Software Engineer ContributionProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695071(783-794)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695071
Show More Cited By

Index Terms

Optimizing Reciprocal Rank with Bayesian Average for improved Next Item Recommendation
1. Information systems
  1. Information retrieval
    1. Retrieval models and ranking
      1. Learning to rank
    2. Retrieval tasks and goals
      1. Recommender systems

Recommendations

An Adaptive Hot Ranking Algorithm for Popular Item Recommendation in the Express Industry
Cognitive Computing - ICCC 2024
Abstract
Within the scope of recommendation systems, content-based and collaborative filtering recommendations encounter difficulties when dealing with the cold start, particularly when considering the temporal evolution of item popularity. These ...
Understanding item consumption orders for right-order next-item recommendation

Although the relevance problem in recommender systems, which typically refers to the similarity between the preference of the user and the items the system recommends, has been well studied, the issue of making recommendations in right orders has barely ...
Using a trust network to improve top-N recommendation
RecSys '09: Proceedings of the third ACM conference on Recommender systems

Top-N item recommendation is one of the important tasks of recommenders. Collaborative filtering is the most popular approach to building recommender systems which can predict ratings for a given user and item. Collaborative filtering can be extended ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SIGIR '23: Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 2023

3567 pages

ISBN:9781450394086

DOI:10.1145/3539618

General Chairs:
Hsin-Hsi Chen
National Taiwan University
,
Wei-Jou (Edward) Duh
National Taiwan University
,
Hen-Hsen Huang
Academia Sinica
,
Program Chairs:
Makoto P. Kato
Spotify
,
Josiane Mothe
Universite de Toulouse
,
Barbara Poblete
University of Chile and Amazon Visiting Academic

Copyright © 2023 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGIR: ACM Special Interest Group on Information Retrieval

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 18 July 2023

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Short-paper

Funding Sources

the National Key Research and Development Plan of China
the Innovation Found from the Engineering Research Center of Integration and Application of Digital Learning Technology, Ministry of Education

Conference

SIGIR '23

Sponsor:

SIGIR

SIGIR '23: The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval

July 23 - 27, 2023

Taipei, Taiwan

Acceptance Rates

Overall Acceptance Rate 792 of 3,983 submissions, 20%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
228
Total Downloads

Downloads (Last 12 months)94
Downloads (Last 6 weeks)9

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Lu XWu J(2025)Combining association-rule-guided sequence augmentation with listwise contrastive learning for session-based recommendationInformation Processing & Management10.1016/j.ipm.2024.10399962:3(103999)Online publication date: May-2025
https://doi.org/10.1016/j.ipm.2024.103999
Wei LLi YWang WZhu Y(2024)Enhancing Knowledge-Concept Recommendations with Heterogeneous Graph-Contrastive LearningMathematics10.3390/math1215232412:15(2324)Online publication date: 25-Jul-2024
https://doi.org/10.3390/math12152324
Li YZhang HJin YRen ZDong LLyu JYang LLo DShao DFilkov VRay BZhou M(2024)An Explainable Automated Model for Measuring Software Engineer ContributionProceedings of the 39th IEEE/ACM International Conference on Automated Software Engineering10.1145/3691620.3695071(783-794)Online publication date: 27-Oct-2024
https://dl.acm.org/doi/10.1145/3691620.3695071
Li YWu JSerra ESpezzano F(2024)Improving Prompt-based News Recommendation with Individual Template and Customized AnswerProceedings of the 33rd ACM International Conference on Information and Knowledge Management10.1145/3627673.3679945(3887-3891)Online publication date: 21-Oct-2024
https://dl.acm.org/doi/10.1145/3627673.3679945
Siwi SWiharja KHafizh R(2024)Transforming Indonesian Geography Education Books Into Knowledge Graphs Using ChatGPT LLMs2024 12th International Conference on Information and Communication Technology (ICoICT)10.1109/ICoICT61617.2024.10698487(50-56)Online publication date: 7-Aug-2024
https://doi.org/10.1109/ICoICT61617.2024.10698487
Li BZeng QRen PChen HGeng YZhang L(2024)An Adaptive Hot Ranking Algorithm for Popular Item Recommendation in the Express IndustryCognitive Computing - ICCC 202410.1007/978-3-031-77954-1_5(71-87)Online publication date: 16-Nov-2024
https://dl.acm.org/doi/10.1007/978-3-031-77954-1_5

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten