Skip to main content

Advertisement

SpringerLink
Log in

MIN: multi-dimensional interest network for click-through rate prediction

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Click-through rate (CTR) prediction is a critical task in recommender systems and online advertising systems. The extensive collection of behavior data has become popular for building prediction models by capturing user interests from behavior sequences. There are two types of entities involved in behavior sequences, users and items, which form three kinds of relationships: user-to-user, user-to-item, and item-to-item. Most related work focuses on only one or two of these relationships, often ignoring the association between users, which also helps discover potential user interests. In this paper, we consider all three relationships useful and propose a Multi-dimensional Interest Network (MIN) to focus on their impact on CTR prediction simultaneously. It consists of three sub-networks that capture users’ preferences regarding group interests and individual interests. Specifically, the u-u sub-network models the relationship between the target user and those who have clicked on the target item. It takes user representations learned from behavior sequences via transformer as input. Two other sub-networks capture the individual interest of the target user. The u-i sub-network models the relationship between the target user and the target item. The i-i sub-network models the relationship between the target item and the items the target user has interacted with in the past time. Extensive evaluations on the real datasets show that our MIN model outperforms the state-of-the-art solutions in prediction accuracy (\(+\) 5.0% in AUC and − 17.2% in Logloss, averagely). The ablation experiments also validate that each sub-network in MIN helps with improving the CTR prediction performance by using the u-u sub-network playing a more critical role. The source code is available at https://github.com/cocolixiao/MIN.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

Notes

  1. http://jmcauley.ucsd.edu/data/amazon/.

References

  1. Zhang W, Du T, Wang J (2016) Deep learning over multi-field categorical data, In: Proceedings of the European conference on information retrieval (ECIR), pp 45–57

  2. Guo W, Su R, Tan R, Guo H, Zhang Y, Liu Z, Tang R, He X (2021) Dual graph enhanced embedding neural network for ctr prediction, In: Proceedings of the 27th ACM SIGKDD conference on knowledge discovery and data mining (SIGKDD), pp 496–504

  3. Yan C, Chen Y, Wan Y, Wang P (2021) Modeling low-and high-order feature interactions with fm and self-attention network. Appl Intell 51(6):3189–3201

    Article  Google Scholar 

  4. Yan C, Li X, Chen Y, Zhang Y (2022) Jointctr: a joint ctr prediction framework combining feature interaction and sequential behavior learning. Appl Intell 52(4):4701–4714

    Article  Google Scholar 

  5. Richardson M, Dominowska E, Ragno R (2007) Predicting clicks: estimating the click-through rate for new ads, In: Proceedings of the 16th international conference on world wide web (WWW), pp 521–530

  6. Rendle S (2012) Factorization machines with libfm. ACM Trans Intell Syst Technol TIST 3(3):1–22

    Article  Google Scholar 

  7. Juan Y, Zhuang Y, Chin WS, Lin CJ (2016) Field-aware factorization machines for ctr prediction, In: Proceedings of the 10th ACM conference on recommender systems (RecSys), pp 43–50

  8. Cheng H-T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M et al (2016) Wide & deep learning for recommender systems, In: Proceedings of the 1st workshop on deep learning for recommender systems (DLRS), pp 7–10

  9. Guo H, Tang R, Ye Y, Li Z, He X (2017) Deepfm: a factorization-machine based neural network for ctr prediction, In: Proceedings of the 26th international joint conference on artificial intelligence (IJCAI), pp 1725–1731

  10. Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction, In: Proceedings of the 16th international conference on data mining (ICDM), pp 1149–1154

  11. Wang R, Fu B, Fu G, Wang M (2017) Deep & cross network for ad click predictions, In: Proceedings of the ADKDD’17, pp 1–7

  12. Lian J, Zhou X, Zhang F, Chen Z, Xie X, Sun G (2018) xdeepfm: Combining explicit and implicit feature interactions for recommender systems, In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining (SIGKDD), pp 1754–1763

  13. Li Z, Cui Z, Wu S, Zhang X, Wang L (2019) Fi-gnn: Modeling feature interactions via graph neural networks for ctr prediction, In: Proceedings of the 28th ACM international conference on information and knowledge management (CIKM), pp 539–548

  14. Liu Q, Yu F, Wu S, Wang L (2015) A convolutional click prediction model, In: Proceedings of the 24th ACM International on conference on information and knowledge management (CIKM), pp 1743–1746

  15. Liu B, Tang R, Chen Y, Yu J, Guo H, Zhang Y (2019) Feature generation by convolutional neural network for click-through rate prediction, In: Proceedings of the conference on world wide web (WWW), pp 1119–1129

  16. Wu S, Yu F, Yu X, Liu Q, Wang L, Tan T, Shao J, Huang F (2020) Tfnet: Multi-semantic feature interaction for ctr prediction, In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 1885–1888

  17. Xiao J, Ye H, He X, Zhang H, Wu F, Chua TS (2017) Attentional factorization machines: Learning the weight of feature interactions via attention networks, In: Proceedings of the 26th international joint conference on artificial intelligence (IJCAI), pp 3119–3125

  18. Huang T, Zhang Z, Zhang J (2019) Fibinet: combining feature importance and bilinear feature interaction for click-through rate prediction, In: Proceedings of the 13th ACM conference on recommender systems (RecSys), pp 169–177

  19. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks, In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7132–7141

  20. Zhang W, Qin J, Guo W, Tang R, He X (2021) Deep learning for click-through rate estimation, In: Proceedings of the 30th international joint conference on artificial intelligence (IJCAI), pp 4695–4703

  21. Zhou G, Zhu X, Song C, Fan Y, Zhu H, Ma X, Yan Y, Jin J, Li H, Gai K (2018) Deep interest network for click-through rate prediction, In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery and data mining (SIGKDD), pp 1059–1068

  22. Zhou G, Mou N, Fan Y, Pi Q, Bian W, Zhou C, Zhu X, Gai K (2019) Deep interest evolution network for click-through rate prediction, In: Proceedings of the AAAI conference on artificial intelligence (AAAI), pp 5941–5948

  23. Feng Y, Lv F, Shen W, Wang M, Sun F, Zhu Y, Yang K (2019) Deep session interest network for click-through rate prediction, In: Proceedings of the 28th international joint conference on artificial intelligence (IJCAI), pp 2301–2307

  24. Huang Z, Tao M, Zhang B (2021) Deep user match network for click-through rate prediction, In: Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 1890–1894

  25. Lyu Z, Dong Y, Huo C, Ren W (2020) Deep match to rank model for personalized click-through rate prediction, In: Proceedings of the AAAI conference on artificial intelligence (AAAI), pp 156–163

  26. Zhang J, Lin F, Yang C, Wang W (2022) Deep multi-representational item network for ctr prediction, In: Proceedings of the 45th international ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 2277–2281

  27. Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations, In: Proceedings of the 10th ACM conference on recommender systems (RecSys), pp 191–198

  28. Qin J, Zhang W, Wu X, Jin J, Fang Y, Yu Y (2020) User behavior retrieval for click-through rate prediction, In: Proceedings of the 43rd international ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 2347–2356

  29. Xu E, Yu Z, Guo B, Cui H (2021) Core interest network for click-through rate prediction. ACM Trans Knowl Discov Data TKDD 15(2):1–16

    Article  Google Scholar 

  30. Ying H, Zhuang F, Zhang F, Liu Y, Xu G, Xie X, Xiong H, Wu J (2018) Sequential recommender system based on hierarchical attention network, In: Proceedings of the international joint conference on artificial intelligence (IJCAI), pp 3926–3932

  31. Hidasi B, Karatzoglou A, Baltrunas L, Tikk D (2016) Session-based recommendations with recurrent neural networks, In: Proceedings of the 4th international conference on learning representations (ICLR), pp 1–10

  32. Tang J, Wang K (2018) Personalized top-n sequential recommendation via convolutional sequence embedding, In: Proceedings of the 11th ACM international conference on web search and data mining (WSDM), pp 565–573

  33. Xiao Z, Yang L, Jiang W, Wei Y, Hu Y, Wang H (2020) Deep multi-interest network for click-through rate prediction, In: Proceedings of the 29th ACM international conference on information and knowledge management (CIKM), pp 2265–2268

  34. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser Ł, Polosukhin I (2017) Attention is all you need, In: Proceedings of the 2017 conference on advances in neural information processing systems (NIPS), pp 5999–6009

  35. Chen Q, Zhao H, Li W, Huang P, Ou W (2019) Behavior sequence transformer for e-commerce recommendation in alibaba, In: Proceedings of the 1st international workshop on deep learning practice for high-dimensional sparse data, pp 1–4

  36. Su X, Khoshgoftaar TM (2009) A survey of collaborative filtering techniques. Adv Artif Intell 2009:1–19

    Article  Google Scholar 

  37. Sarwar B, Karypis G, Konstan J, Riedl J (2001) Item-based collaborative filtering recommendation algorithms, In: Proceedings of the 10th international conference on world wide web (WWW), pp.285–295

  38. Zhao ZD, Shang MS (2010) User-based collaborative-filtering recommendation algorithms on hadoop, In: Proceedings of the 3rd international conference on knowledge discovery and data mining (WKDD), pp 478–481

  39. Koren Y (2008) Factorization meets the neighborhood: a multifaceted collaborative filtering model, In: Proceedings of the 14th ACM SIGKDD international conference on knowledge discovery and data nining (SIGKDD), pp 426–434

  40. Koren Y, Bell R, Volinsky C (2009) Matrix factorization techniques for recommender systems. Computer 42(8):30–37

    Article  Google Scholar 

  41. Wang X, He X, Wang M, Feng F, Chua T-S (2019) Neural graph collaborative filtering, In: Proceedings of the 42nd International ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 165–174

  42. Lin Z, Tian C, Hou Y, Zhao WX (2022) Improving graph collaborative filtering with neighborhood-enriched contrastive learning, In: Proceedings of the ACM web conference (WWW), pp 2320–2329

  43. Wolf T, Debut L, Sanh V, Chaumond J, Delangue C, Moi A, Cistac P, Rault T, Louf R, Funtowicz M et al (2020) Transformers: State-of-the-art natural language processing, In: Proceedings of the 2020 conference on empirical methods in natural language processing (EMNLP), pp 38–45

  44. Han K, Xiao A, Wu E, Guo J, Xu C, Wang Y (2021) Transformer in transformer, In: Proceedings of the conference on neural information processing systems (NeurIPS), pp 15908–15919

  45. Tang J, Belletti F, Jain S, Chen M, Beutel A, Xu C, Chi EH (2019) Towards neural mixture recommender for long range dependent user sequences, In: Proceedings of the world wide web conference (WWW), pp 1782–1793

Download references

Acknowledgements

This work is partly supported by the Shanghai Sailing Program (21YF1401300).

Author information

Authors and Affiliations

  1. Donghua University, Shanghai, 201620, China

    Cairong Yan, Xiaoke Li, Yanting Zhang & Zijian Wang

  2. Shanghai University, Shanghai, 200444, China

    Yongquan Wan

  3. Shanghai Jian Qiao University, Shanghai, 201306, China

    Yongquan Wan

Authors
  1. Cairong Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Xiaoke Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yanting Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zijian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yongquan Wan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

XL: Software, Formal analysis, Investigation, Data curation, Writing-original draft. CY: Conceptualization, Methodology, Writing-reviewing & editing. YZ: Visualization, Funding acquisition. ZW: Commenting on the proposed idea. YW: Validation, Writing-reviewing & editing.

Corresponding authors

Correspondence to Cairong Yan or Xiaoke Li.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Yan, C., Li, X., Zhang, Y. et al. MIN: multi-dimensional interest network for click-through rate prediction. Knowl Inf Syst 65, 3945–3965 (2023). https://doi.org/10.1007/s10115-023-01885-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-023-01885-8

Keywords

Search

Navigation