skip to main content
10.1145/3477495.3531845acmconferencesArticle/Chapter ViewAbstractPublication PagesirConference Proceedingsconference-collections
short-paper

Deep Multi-Representational Item Network for CTR Prediction

Published: 07 July 2022 Publication History

Abstract

Click-through rate (CTR) prediction is essential in the modelling of a recommender system. Previous studies mainly focus on user behavior modelling, while few of them consider candidate item representations. This makes the models strongly dependent on user representations, and less effective when user behavior is sparse. Furthermore, most existing works regard the candidate item as one fixed embedding and ignore the multi-representational characteristics of the item. To handle the above issues, we propose a Deep multi-Representational Item NetworK (DRINK) for CTR prediction. Specifically, to tackle the sparse user behavior problem, we construct a sequence of interacting users and timestamps to represent the candidate item; to dynamically capture the characteristics of the item, we propose a transformer-based multi-representational item network consisting of a multi-CLS representation submodule and contextualized global item representation submodule. In addition, we propose to decouple the time information and item behavior to avoid information overwhelming. Outputs of the above components are concatenated and fed into a MLP layer to fit the CTR. We conduct extensive experiments on real-world datasets of Amazon and the results demonstrate the effectiveness of the proposed model.

Supplementary Material

MP4 File (SIGIR22-sp1750.mp4)
Click-through rate (CTR) prediction is essential in the modelling of a recommender system. Previous studies mainly focus on user behavior modelling, while few of them consider candidate item representations. Furthermore, most existing works regard the candidate item as one fixed embedding and ignore the multi-representational characteristics of the item. To handle the above issues, we propose a Deep multi-Representational Item NetworK (DRINK) for CTR prediction. Specifically, to tackle the sparse user behavior problem, we construct a sequence of interacting users and timestamps to represent the candidate item; to dynamically capture the characteristics of the item, we propose a transformer-based multi-representational item network consisting of a multi-CLS representation submodule and contextualized global item representation submodule.

References

[1]
Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, et al. 2016. Tensorflow: Large-scale machine learning on heterogeneous distributed systems. arXiv preprint arXiv:1603.04467 (2016).
[2]
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016. Wide & deep learning for recommender systems. In Proceedings of the 1st workshop on deep learning for recommender systems. 7--10.
[3]
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555 (2014).
[4]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805 (2018).
[5]
Hongliang Fei, Jingyuan Zhang, Xingxuan Zhou, Junhao Zhao, Xinyang Qi, and Ping Li. 2021. GemNN: gating-enhanced multi-task neural networks with feature interaction learning for CTR prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2166--2171.
[6]
Alex Graves. 2013. Generating sequences with recurrent neural networks. arXiv preprint arXiv:1308.0850 (2013).
[7]
Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based recommendations with recurrent neural networks. In International Conference on Learning Representations (ICLR).
[8]
Jianqiang Huang, Ke Hu, Qingtao Tang, Mingjian Chen, Yi Qi, Jia Cheng, and Jun Lei. 2021. Deep Position-wise Interaction Network for CTR Prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1885--1889.
[9]
Zai Huang, Mingyuan Tao, and Bufeng Zhang. 2021. Deep User Match Network for Click-Through Rate Prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 1890-- 1894.
[10]
Wei Jiang, Fangquan Lin, Jihai Zhang, Cheng Yang, Hanwei Zhang, and Ziqiang Cui. 2021. Dynamic Sequential Recommendation: Decoupling User Intent from Temporal Context. In ICDMW. IEEE, 18--26.
[11]
Wang-Cheng Kang and Julian McAuley. 2018. Self-attentive sequential recommendation. In 2018 IEEE 18th International Conference on Data Mining (ICDM). IEEE, 197--206.
[12]
Mikhail Khalilov and Alexey Timoveev. 2021. Performance analysis of CUDA, OpenACC and OpenMP programming models on TESLA V100 GPU. In Journal of Physics: Conference Series, Vol. 1740. IOP Publishing, 012056.
[13]
Yehuda Koren. 2008. Factorization meets the neighborhood: a multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining. 426--434.
[14]
Chao Li, Zhiyuan Liu, Mengmeng Wu, Yuchi Xu, Huan Zhao, Pipei Huang, Guoliang Kang, Qiwei Chen, Wei Li, and Dik Lun Lee. 2019. Multi-interest network with dynamic routing for recommendation at Tmall. In Proceedings of the 28th ACM International Conference on Information and Knowledge Management. 2615--2623.
[15]
Feng Li, Bencheng Yan, Qingqing Long, Pengjie Wang, Wei Lin, Jian Xu, and Bo Zheng. 2021. Explicit Semantic Cross Feature Learning via Pre-trained Graph Neural Networks for CTR Prediction. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. 2161--2165.
[16]
Hao Li, Huichuan Duan, Yuanjie Zheng, Qianqian Wang, and Yu Wang. 2020. A CTR prediction model based on user interest via attention mechanism. Applied Intelligence 50, 4 (2020), 1192--1203.
[17]
Jing Li, Pengjie Ren, and Zhumin Chen. 2017. Neural Attentive Session-based Recommendation. In Proceedings of the 26th ACM International Conference on Information & Knowledge Management. ACM.
[18]
Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware selfattention for sequential recommendation. In Proceedings of the 13th International Conference on Web Search and Data Mining. 322--330.
[19]
Xiang Li, Chao Wang, Bin Tong, Jiwei Tan, Xiaoyi Zeng, and Tao Zhuang. 2020. Deep Time-Aware Item Evolution Network for Click-Through Rate Prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 785--794.
[20]
Fangquan Lin, Wei Jiang, Jihai Zhang, and Cheng Yang. 2021. Dynamic popularityaware contrastive learning for recommendation. In ACML. PMLR, 964--968.
[21]
Ze Lyu, Yu Dong, Chengfu Huo, and Weijun Ren. 2020. Deep match to rank model for personalized click-through rate prediction. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 156--163.
[22]
Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton Van Den Hengel. 2015. Image-based recommendations on styles and substitutes. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval. 43--52.
[23]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Neural Information Processing Systems (NeurIPS). 3111--3119.
[24]
Jianbo Ouyang, Hui Wu, Min Wang, Wengang Zhou, and Houqiang Li. 2021. Contextual Similarity Aggregation with Self-attention for Visual Re-ranking. Advances in Neural Information Processing Systems 34 (2021).
[25]
Yanru Qu, Han Cai, Kan Ren, Weinan Zhang, Yong Yu, Ying Wen, and Jun Wang. 2016. Product-based neural networks for user response prediction. In 2016 IEEE 16th International Conference on Data Mining (ICDM). IEEE, 1149--1154.
[26]
Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. BERT4Rec: Sequential recommendation with bidirectional encoder representations from transformer. In Conference on Information and Knowledge Management. 1441--1450.
[27]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Neural Information Processing Systems (NeurIPS). 5998--6008.
[28]
Qianqian Wang, Pu Huang, Shuning Xing, Xiaohui Zhao, et al. 2019. A hierarchical attention model for CTR prediction based on user interest. IEEE Systems Journal 14, 3 (2019), 4015--4024.
[29]
Hui Wu, Min Wang, Wengang Zhou, Yang Hu, and Houqiang Li. 2021. Learning Token-based Representation for Image Retrieval. arXiv preprint arXiv:2112.06159 (2021).
[30]
Zhibo Xiao, Luwei Yang, Wen Jiang, Yi Wei, Yi Hu, and Hao Wang. 2020. Deep Multi-Interest Network for Click-through Rate Prediction. In Proceedings of the 29th ACM International Conference on Information & Knowledge Management. 2265--2268.
[31]
Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the AAAI conference on artificial intelligence, Vol. 33. 5941--5948.
[32]
Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018. Deep interest network for click-through rate prediction. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. 1059--1068.

Cited By

View all
  • (2025)High-Reputation Food Formulas: A Heterogeneous Information Network Representation and Semantic Analysis ApproachApplied Sciences10.3390/app1505237515:5(2375)Online publication date: 23-Feb-2025
  • (2024)Cold-start recommendation by personalized embedding region elicitationProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702808(2766-2786)Online publication date: 15-Jul-2024
  • (2024)ALGIN: Adaptive Local and Global Interests Network for Click-Through Rate Prediction2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10830951(2186-2193)Online publication date: 6-Oct-2024
  • Show More Cited By

Index Terms

  1. Deep Multi-Representational Item Network for CTR Prediction

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    SIGIR '22: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
    July 2022
    3569 pages
    ISBN:9781450387323
    DOI:10.1145/3477495
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 07 July 2022

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. ctr prediction
    2. item behavior representation
    3. recommendation

    Qualifiers

    • Short-paper

    Conference

    SIGIR '22
    Sponsor:

    Acceptance Rates

    Overall Acceptance Rate 792 of 3,983 submissions, 20%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)77
    • Downloads (Last 6 weeks)4
    Reflects downloads up to 28 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)High-Reputation Food Formulas: A Heterogeneous Information Network Representation and Semantic Analysis ApproachApplied Sciences10.3390/app1505237515:5(2375)Online publication date: 23-Feb-2025
    • (2024)Cold-start recommendation by personalized embedding region elicitationProceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence10.5555/3702676.3702808(2766-2786)Online publication date: 15-Jul-2024
    • (2024)ALGIN: Adaptive Local and Global Interests Network for Click-Through Rate Prediction2024 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC54092.2024.10830951(2186-2193)Online publication date: 6-Oct-2024
    • (2024)DISS-CF: Direct Item Session Similarity Enhanced Collaborative Filtering Method for Recommendation2024 IEEE International Conference on Web Services (ICWS)10.1109/ICWS62655.2024.00054(320-329)Online publication date: 7-Jul-2024
    • (2024)Interactive Attention-Based Capsule Network for Click-Through Rate PredictionIEEE Access10.1109/ACCESS.2024.344478712(170335-170345)Online publication date: 2024
    • (2023)Combining Forecasting and Multi-Agent Reinforcement Learning Techniques on Power Grid Scheduling Task2023 IEEE 2nd International Conference on Electrical Engineering, Big Data and Algorithms (EEBDA)10.1109/EEBDA56825.2023.10090669(1576-1580)Online publication date: 24-Feb-2023
    • (2023)Item Multi-Information Evolution Network for Click-Through Rate Prediction2023 26th International Conference on Computer Supported Cooperative Work in Design (CSCWD)10.1109/CSCWD57460.2023.10152622(285-290)Online publication date: 24-May-2023
    • (2023)Evolving Interest with Feature Co-action Network for CTR PredictionData Science and Engineering10.1007/s41019-023-00217-88:4(344-356)Online publication date: 2-Sep-2023
    • (2023)MIN: multi-dimensional interest network for click-through rate predictionKnowledge and Information Systems10.1007/s10115-023-01885-865:10(3945-3965)Online publication date: 12-May-2023
    • (2023)Attention-Based Feature Interaction Deep Factorization Machine for CTR PredictionArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44204-9_5(49-60)Online publication date: 26-Sep-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media