Skip to main content
Springer Nature Link
Log in

HoINT: Learning Explicit and Implicit High-order Feature Interactions for Click-through Rate Prediction

  • Published:
Neural Processing Letters Aims and scope Submit manuscript

Abstract

Click-through rate (CTR) prediction is a research hotspot in the field of recommendation systems and online advertising. Because of the diversity, large-scale, and high real-time characteristics of Internet data, manual feature interaction is almost impossible. Although existing models can learn feature interactions without manual feature engineering, few studies attempt to learn both explicit and implicit high-order feature interactions simultaneously. In order to effectively capture explicit and implicit high-order feature interactions, and automatically identify important feature interactions in a larger feature interaction space, we construct a parallel model that integrates a multi-head self-attention network and a Bilinear-DNN module (HoINT), which can learn high-order feature interactions automatically in both explicit and implicit ways. Sufficient experiments on four real-world datasets indicate that the HoINT model proposed is better than the most typical and advanced models, and the relative contributions of different components of the model are assessed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles, news and stories from top researchers in related subjects.

References

  1. Jian M et al (2020) Content-based bipartite user-image correlation for image recommendation. Neural Process Lett 52(2):1445–1459

    Article  Google Scholar 

  2. Guo S et al (2020) Developer activity motivated bug triaging: via convolutional neural network. Neural Process Lett 51(3):2589–2606

    Article  Google Scholar 

  3. Pi Q, Bian W, Zhou G, et al. (2019) Practice on long sequential user behavior modeling for click-through rate prediction. In: the 25th ACM SIGKDD international conference ACM

  4. He X, Pan J, Jin O et al (2014) Practical lessons from predicting clicks on ads at facebook. In: proceedings of the 8th international workshop on data mining for online advertising (ADKDD) - in conjunction with SIGKDD

  5. Gai K, Zhu X, Li H et al (2017) Learning piece-wise linear models from large scale data for ad click prediction. arXiv:1704.05194

  6. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Adv Neural Inform Process Syst 25:1097–1105

    Google Scholar 

  7. Cheng H, Koc L, Harmsen J et al (2016) Wide & deep learning for recommender systems. In: proceedings of the 1st workshop on deep learning for recommender systems (DLRS) - in conjunction with RecSys

  8. Lian J, Zhou X, Zhang F, et al. (2018) xDeepFM: combining explicit and implicit feature interactions for recommender systems

  9. Guo H, Tang R, Ye Y, et al. (2017) DeepFM: a factorization-machine based neural network for CTR prediction

  10. Zhang W, Du T, Wang J (2016) deep learning over multi-field categorical data. In: European conference on information retrieval. Springer, Cham

  11. Wang R, Fu G, Fu B et al (2017) Deep & cross network for ad click predictions. In: proceedings of the 2017 AdKDD and TargetAd - In conjunction with ACM SIGKDD

  12. Mcmahan HB, Holt G, Sculley D, et al. (2013) Ad click prediction: a view from the trenches. In: proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM

  13. Chang Y, Hsieh C, Chang K et al (2010) Training and testing lowdegree polynomial data mappings via linear SVM. J Mach Learn Res 11(4):1471–1490

    MathSciNet  MATH  Google Scholar 

  14. Rendle S (2011) Factorization machines with libFM. ACM Trans Intel Syst Technol 3(3):57

    Google Scholar 

  15. Juan Y, Zhuang Y, Chin W et al (2016) Field-aware factorization machines for CTR prediction. In: proceedings of the 10th ACM conference on recommender systems

  16. Qu Y, Cai H, Ren K et al (2016) Product-based neural networks for user response prediction. In: proceedings of the 16th IEEE international conference on data mining (ICDM)

  17. Guo W, Tang R, Guo H, Han J, Yang W, & Zhang Y. (2019).Order-aware Embedding Neural Network for CTR Prediction. In: proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval - SIGIR’19

  18. Yi YA, Bx A, Ss A et al (2020) Operation-aware neural networks for user response prediction. Neural Netw 121:161–168

    Article  Google Scholar 

  19. Liu B, Tang R, Chen Y, Yu J, Guo H, and Zhang Y (2019) Feature generation by convolutional neural network for click-through rate prediction. In: proceedings of World Wide Web conference (WWW), pp. 1119–1129

  20. He X, Chua TS (2017) Neural factorization machines for sparse predictive analytics. In: proceedings of the 40th international ACM SIGIR conference on research and development in information retrieval (SIGIR)

  21. Xiao J, Ye H, He X et al (2017) Attentional factorization machines: Learning the weight of feature interactions via attention networks. In: proceedings of the 26th international joint conference on artificial intelligence (IJCAI)

  22. Liu B, Zhu C, Li G, et al. (2020) AutoFIS: automatic feature interaction selection in factorization models for click-through rate prediction. In: proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining

  23. Liu M, Cai S, Lai Z et al (2021) A joint learning model for click-through prediction in display advertising. Neurocomputing 445:206–219

    Article  Google Scholar 

  24. Li D et al (2021) Attentive capsule network for click-through rate and conversion rate prediction in online advertising. Knowl Based Syst 211:106522

    Article  Google Scholar 

  25. Zhou G, Song C, Zhu X, (2017) Deep interest network for click-through rate prediction. In: proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, London, United Kingdom, pp. 1059–1068

  26. Feng Y, Lv F, Shen W, Wang M, and Sun F, (2019) Deep session interest network for click-through rate prediction. In: proceedings of IJCAI conference artificial intellegence

  27. Huang T, Zhang Z, Zhang J (2019) FiBiNET: combining feature importance and bilinear feature interaction for click-throug rate prediction

  28. Jie H, Li S, Gang S, et al. (2017) Squeeze-and-excitation networks. IEEE Trans Pattern Anal Mach Intell, pp 99

  29. Yu Y, Wang Z, and Yuan B, (2019) An input-aware factorization machine for sparse prediction. In: IJCAI international joint conference on artificial intellegence, vol. 2019-Augus, pp. 1466–1472

  30. Dan JB et al (2021) Multi-view feature transfer for click-through rate prediction - ScienceDirect. Inf Sci 546:961–976

    Article  Google Scholar 

  31. Song K et al (2021) Coarse-to-fine: a dual-view attention network for click-through rate prediction. Knowl-Based Syst 216(4):106767

    Article  Google Scholar 

  32. Zhang J, Ma C, Zhong C, Zhao P, Mu X (2022) Multi-scale and multi-channel neural network for click-through rate prediction. Neurocomputing 480:157–168

    Article  Google Scholar 

  33. Abinaya S, Devi M (2021) Enhancing top-N recommendation using stacked autoencoder in context-aware recommender system. Neural Process Lett 53:1865–1888

    Article  Google Scholar 

  34. Vaswani A, Shazeer N, Parmar N et al. (2017) Attention is all you need. In: proceedings of the conference on advances in neural information processing systems. pp. 5998–6008

  35. Yan C, Li X, Chen Y et al (2021) JointCTR: a joint CTR prediction framework combining feature interaction and sequential behavior learning. Appl Intell 52(4):4701–4714

    Article  Google Scholar 

  36. Gai K, Zhu X, and Li H, (2017) Learning piece-wise linear models from large scale data for ad click prediction. arXiv:1704.05194. [Online]. Available: https://arxiv.org/abs/1704.05194

  37. Wang, R., et al. (2020) DCN V2: Improved deep & cross network and practical lessons for web-scale learning to rank systems. In: proceedings of the web conference

  38. Song W, Shi C, Xiao Z (2018) Autoint: Automatic feature interaction learning via self-attentive neural networks. In: proceedings of the 28th ACM international conference on information and knowledge management, pp. 1161–1170

  39. Yan C, Chen Y, Wan Y et al (2020) Modeling low- and high-order feature interactions with FM and self-attention network. Appl Intell 4:1–13

    Google Scholar 

  40. Y. Gal and Z. Ghahramani, Dropout as a Bayesian approximation: representing model uncertainty in deep learning. In: proceedings of the international conference on machine learning, New York, NY, Jun. 2016, pp. 1050–1059

Download references

Acknowledgements

This work was supported by the National Science Foundation of China under Grant 61472095 and the Natural Science Foundation of Heilongjiang Province under Grant LH2020F023.

Author information

Authors and Affiliations

  1. College of Computer Science and Technology, Harbin Engineering University, Harbin, 150001, China

    Hongbin Dong & Xiaowei Wang

Authors
  1. Hongbin Dong
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Xiaowei Wang
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Xiaowei Wang.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dong, H., Wang, X. HoINT: Learning Explicit and Implicit High-order Feature Interactions for Click-through Rate Prediction. Neural Process Lett 55, 401–421 (2023). https://doi.org/10.1007/s11063-022-10889-4

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11063-022-10889-4

Keywords