Skip to main content
SpringerLink
Log in

CFF: combining interactive features and user interest features for click-through rate prediction

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Click-through rate is a central issue in ad recommendation and has recently received extensive research attention in academia and industry. Research shows that the accuracy of prediction results in CTR prediction is closely related to interactive features and user interest features. However, existing models usually focus on one aspect of features, i.e., interactive features or interest features, and few studies have attempted to learn both interactive features and interest features simultaneously. In this paper, a novel model called CFF as an abbreviation for Combining interactive Features and interest Features is proposed to learn interactive features and user interest features simultaneously. To efficiently learn fine-grained interactive features, an attention-based squeeze equal interaction network (ASENet) is constructed to select salient feature information at the level of equal interactive features. A bi-directional attention-target item gated recurrent unit (Bi-ATGRU) is designed to learn the dependencies between user interests and items. Specifically, it refines and integrates interest features by incorporating context information, historical behaviors, and target item. Extensive experiments on four public datasets indicate CFF outperforms other baselines in terms of evaluation metrics (the Logloss decreases by 1.97% on Frappe and 1.85% on MovieLens).

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

The datasets and code during the current study are available at the footnote link above.

Notes

  1. User interest features are the user’s personal tendency and preference for items. In this paper, latent interests and user preferences are the synonyms of user interests.

  2. Interactive features refer to dynamic patterns that emerge when different elements within a system mutually influence each other.

  3. http://www.libfm.org/.

  4. http://baltrunas.info/research-menu/frappe.

  5. http://grouplens.org/datasets/movielens/latest.

  6. https://www.kaggle.com/c/criteo-display-ad-challenge.

  7. https://www.kaggle.com/c/avazu-ctr-prediction.

  8. https://github.com/NarLiDao/First.

References

  1. Yang Y, Yang YC, Jansen BJ, Lalmas M (2017) Computational advertising: a paradigm shift for advertising and marketing? IEEE Intell Syst 32(3):3–6

    Article  Google Scholar 

  2. Feng J, Bian J, Wang T, Chen W, Zhu X, Liu T-Y (2014) Sampling dilemma: towards effective data sampling for click prediction in sponsored search. In: Proceedings of the 7th ACM International Conference on Web Search and Data Mining, pp 103–112

  3. Qiu R, Ji W (2021) An embedded bandit algorithm based on agent evolution for cold-start problem. Int J Crowd Sci 5(3):228–238

    Article  Google Scholar 

  4. Zhang Q, Liu J, Dai Y, Qi Y, Yuan Y, Zheng K, Huang F, Tan X (2022) Multi-task fusion via reinforcement learning for long-term user satisfaction in recommender systems. In: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp 4510–4520

  5. Lin S, Yu Y, Ji X, Zhou T, He H, Sang Z, Jia J, Cao G, Hu N (2022) Spatiotemporal-enhanced network for click-through rate prediction in location-based services. arXiv preprint arXiv:2209.09427

  6. Zhang S, Fu Q, Xiao W (2017) Advertisement click-through rate prediction based on the weighted-elm and adaboost algorithm. Sci Programm 2017

  7. Liu W, Tang R, Li J, Yu J, Guo H, He X, Zhang S (2018) Field-aware probabilistic embedding neural network for ctr prediction. In: Proceedings of the 12th ACM Conference on Recommender Systems, pp 412–416

  8. McMahan HB, Holt G, Sculley D, Young M, Ebner D, Grady J, Nie L, Phillips T, Davydov E, Golovin D, et al. (2013) Ad click prediction: a view from the trenches. In: Proceedings of the 19th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp 1222–1230

  9. Tan M, Zhou J, Peng Z, Yu J, Tang F (2020) Fine-grained image classification with factorized deep user click feature. Inf Process Manag 57(3):102186

    Article  Google Scholar 

  10. Yan L, Li W-J, Xue G-R, Han D (2014) Coupled group lasso for web-scale ctr prediction in display advertising. In: Proceedings of the 31st International Conference on Machine Learning, pp 802–810. PMLR

  11. Rendle S (2010) Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE

  12. Rendle S (2012) Factorization machines with libfm. ACM Trans Intell Syst Technol (TIST) 3(3):1–22

    Article  Google Scholar 

  13. Juan Y, Zhuang Y, Chin W-S, Lin C-J (2016) Field-aware factorization machines for ctr prediction. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 43–50

  14. Covington P, Adams J, Sargin E (2016) Deep neural networks for youtube recommendations. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp 191–198

  15. Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp 1149–1154. IEEE

  16. Zhang W, Du T, Wang J (2016) Deep learning over multi-field categorical data. In: European Conference on Information Retrieval, pp. 45–57 . Springer

  17. Lian J, Zhou X, Zhang F, Chen Z, Xie X, Sun G (2018) xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp 1754–1763

  18. Wang Q, Huang P, Xing S, Zhao X et al (2019) A hierarchical attention model for ctr prediction based on user interest. IEEE Syst J 14(3):4015–4024

    Article  Google Scholar 

  19. Zhou G, Mou N, Fan Y, Pi Q, Bian W, Zhou C, Zhu X, Gai K (2019) Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 33, pp 5941–5948

  20. Wang Q, Liu F, Xing S, Zhao X (2019) Research on ctr prediction based on stacked autoencoder. Appl Intell 49(8):2970–2981

    Article  Google Scholar 

  21. Bian W, Wu K, Ren L, Pi Q, Zhang Y, Xiao C, Sheng X-R, Zhu Y-N, Chan Z, Mou N, et al. (2022) Can: Feature co-action network for click-through rate prediction. In: Proceedings of the 15th ACM International Conference on Web Search and Data Mining, pp 57–65

  22. Guo H, Tang R, Ye Y, Li Z, He X (2017) Deepfm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247

  23. Song Y, Elkahky AM, He X (2016) Multi-rate deep learning for temporal recommendation. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 909–912

  24. Zhang Y, Dai H, Xu C, Feng J, Wang T, Bian J, Wang B, Liu T-Y (2014) Sequential click prediction for sponsored search with recurrent neural networks. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 28

  25. Huang T, Zhang Z, Zhang J (2019) Fibinet: combining feature importance and bilinear feature interaction for click-through rate prediction. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp 169–177

  26. Luo L, Chen Y, Liu X, Deng Q (2020) Feature aware and bilinear feature equal interaction network for click-through rate prediction. In: Neural Information Processing: 27th International Conference, ICONIP 2020, Bangkok, Thailand, November 23–27, 2020, Proceedings, Part III 27, pp 432–443. Springer

  27. Yang H, Yao L, Cai J, Wang Y, Zhao X (2023) A new interest extraction method based on multi-head attention mechanism for ctr prediction. Knowl Inf Syst 65(8):3337–3352

    Article  Google Scholar 

  28. Xiao J, Ye H, He X, Zhang H, Wu F, Chua T-S (2017) Attentional factorization machines: learning the weight of feature interactions via attention networks. arXiv preprint arXiv:1708.04617

  29. Song W, Shi C, Xiao Z, Duan Z, Xu Y, Zhang M, Tang J (2019) Autoint: automatic feature interaction learning via self-attentive neural networks. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management, pp 1161–1170

  30. Xiao Z, Yang L, Jiang W, Wei Y, Hu Y, Wang H (2020) Deep multi-interest network for click-through rate prediction. In: Proceedings of the 29th ACM International Conference on Information & Knowledge Management, pp 2265–2268

  31. Yu S, Yang C, Jie Z, Shi X (2022) Time-aware attentive click sequence network for click-through rate prediction. In: Proceedings of the 4th International Conference on Big Data Engineering, pp 134–139

  32. Pan J, Xu J, Ruiz AL, Zhao W, Pan S, Sun Y, Lu Q (2018) Field-weighted factorization machines for click-through rate prediction in display advertising. In: Proceedings of the 2018 World Wide Web Conference, pp 1349–1357

  33. Sun Y, Pan J, Zhang A, Flores A (2021) Fm2: field-matrixed factorization machines for recommender systems. In: Proceedings of the Web Conference 2021, pp 2828–2837

  34. Chen L-C, Papandreou G, Kokkinos I, Murphy K, Yuille AL (2017) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  35. Mnih V, Heess N, Graves A, et al (2014) Recurrent models of visual attention. Adv Neural Inf Process Syst 27

  36. Cheng H-T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M, et al. (2016) Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pp 7–10

  37. He X, Chua T-S (2017) Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 355–364

  38. Cheng W, Shen Y, Huang L (2020) Adaptive factorization network: Learning adaptive-order feature interactions. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 34, pp 3609–3616

  39. Li D, Hu B, Chen Q, Wang X, Qi Q, Wang L, Liu H (2021) Attentive capsule network for click-through rate and conversion rate prediction in online advertising. Knowl Based Syst 211:106522

    Article  Google Scholar 

  40. Liu S, Chen D, Shao J (2021) Ada: adaptive depth attention model for click-through rate prediction. In: 2021 International Joint Conference on Neural Networks (IJCNN), pp 1–8. IEEE

  41. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp 7132–7141

  42. Dong H, Wang X (2022) Hoint: Learning explicit and implicit high-order feature interactions for click-through rate prediction. Neural Process Lett, 1–21

  43. Li Z, Cheng W, Chen Y, Chen H, Wang W (2020) Interpretable click-through rate prediction through hierarchical attention. In: Proceedings of the 13th International Conference on Web Search and Data Mining, pp 313–321

  44. Tao Z, Wang X, He X, Huang X, Chua T-S (2020) Hoafm: a high-order attentive factorization machine for ctr prediction. Inf Process Manag 57(6):102076

    Article  Google Scholar 

  45. Yu F, Liu Q, Wu S, Wang L, Tan T (2016) A dynamic recurrent model for next basket recommendation. In: Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 729–732

  46. McAuley J, Targett C, Shi Q, Van Den Hengel A (2015) Image-based recommendations on styles and substitutes. In: Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 43–52

  47. Zhou C, Bai J, Song J, Liu X, Zhao Z, Chen X, Gao J (2018) Atrank: An attention-based user behavior modeling framework for recommendation. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol 32

  48. Ren K, Fang Y, Zhang W, Liu S, Li J, Zhang Y, Yu Y, Wang J (2018) Learning multi-touch conversion attribution with dual-attention mechanisms for online advertising. In: Proceedings of the 27th ACM International Conference on Information and Knowledge Management, pp 1433–1442

  49. Yan C, Li X, Chen Y, Zhang Y (2022) Jointctr: a joint ctr prediction framework combining feature interaction and sequential behavior learning. Appl Intell 52(4):4701–4714

    Article  Google Scholar 

  50. Qin C, Xie J, Jiang Q, Chen X (2023) A novel interest evolution network based on transformer and a gated residual for ctr prediction in display advertising. Neural Computi Appl, 1–16

  51. Zhang W, Han Y, Yi B, Zhang Z (2023) Click-through rate prediction model integrating user interest and multi-head attention mechanism. J Big Data 10(1):11

    Article  Google Scholar 

  52. Xiao Y, He W, Zhu Y, Zhu J (2022) A click-through rate model of e-commerce based on user interest and temporal behavior. Expert Syst Appl 207:117896

    Article  Google Scholar 

  53. Min E, Rong Y, Xu T, Bian Y, Luo D, Lin K, Huang J, Ananiadou S, Zhao P (2022) Neighbour interaction based click-through rate prediction via graph-masked transformer. In: Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp 353–362

Download references

Funding

This work is supported in part by the Natural Science Foundation of Shandong under Grant ZR202011020044, in part by the National Natural Science Foundation of China under Grant 61772321, in part by the Key Research and Development Plan of Shandong Province under Grant 2019GGX101075.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Shandong Normal University, Jinan, 250358, Shandong, China

    Lin Zhang, Fang’ai Liu, Hongchen Wu, Xuqiang Zhuang & Yaoyao Yan

Authors
  1. Lin Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fang’ai Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hongchen Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xuqiang Zhuang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yaoyao Yan
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

LZ contributed to conceptualization, methodology, investigation, writing-review draft & editing. FL contributed to resources, supervision, and funding acquisition. HW contributed to formal analysis, data curation, and funding acquisition. XZ contributed to validation. YY contributed to validation.

Corresponding authors

Correspondence to Fang’ai Liu or Hongchen Wu.

Ethics declarations

Conflict of Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Ethical approval

The work was established, according to the ethical guidelines.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, L., Liu, F., Wu, H. et al. CFF: combining interactive features and user interest features for click-through rate prediction. J Supercomput 80, 3282–3309 (2024). https://doi.org/10.1007/s11227-023-05598-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-023-05598-1

Keywords

Search

Navigation