Skip to main content
SpringerLink
Log in

A CTR prediction model based on user interest via attention mechanism

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Recently, click-through rate (CTR) prediction is a challenge problem in the aspect of online advertising. Some researchers have proposed deep learning-based models that follow a similar embedding and MLP paradigm. However, the corresponding approaches generally ignore the importance of capturing the latent user interest behind user behaviour data. In this paper, we present a novel attentive deep interest-based network model called ADIN. Specifically, we capture the interest sequence in the interest extractor layer, and the auxiliary losses are employed to produce the interest state with the deep supervision. First, we model the dependency between behaviours by using a bidirectional gated recurrent unit (Bi-GRU). Next, we extract the interest evolving process that is related to the target and propose an interest evolving layer. At the same time, attention mechanism is embedded into the sequential structure. Then, the model learns highly non-linear interactions of features based on stack autoencoders. An experiment has been done using four real-world datasets, the proposed model achieves superior performance than the existing state-of-the-art models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Chapelle O, Rosales R, Rosales R (2015) Simple and scalable response prediction for display advertising [M]. ACM

  2. Jahrer M, Toscher A, Lee J Y, et al. (2012) Ensemble of collaborative filtering and feature engineered models for click through rate prediction[C]//KDDCup workshop

  3. Cheng H T, Koc L, Harmsen J, et al. (2016) Wide & Deep Learning for recommender systems[C]// the workshop on deep learning for recommender systems. ACM, 7–10

  4. Najafi-Asadolahi S, Fridgeirsdottir K (2014) Cost-per-click pricing for display advertising[J]. Manuf Serv Oper Manag 16(4):482–497

    Article  Google Scholar 

  5. Richardson M, Dominowska E, Ragno R (2007) "Predicting clicks: estimating the click-through rate for new ads." Proceedings of the 16th international conference on World Wide Web. ACM

  6. Chapelle O, Manavoglu E, Rosales R (2015) Simple and scalable response prediction for display advertising[J]. ACM Trans Intell Syst Technol (TIST) 5(4):61

    Google Scholar 

  7. Chapelle O (2014) "Modeling delayed feedback in display advertising." Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM

  8. McMahan, HB, et al. (2013) "Ad click prediction: a view from the trenches." Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. ACM

  9. Chang YW, Hsieh CJ, Chang KW et al (2010) Training and testing low-degree polynomial data mappings via linear SVM[J]. J Mach Learn Res 11(Apr):1471–1490

    MathSciNet  MATH  Google Scholar 

  10. Wang B (2013) Advertisement click-through rate prediction using multiple criteria linear programming regression model [J]. Proc Comput Sci 17:803–811

    Article  Google Scholar 

  11. Rendle S (2012) "Factorization machines with libfm." ACM Trans Intell Syst Technol (TIST) 3.3: 57

    Article  Google Scholar 

  12. Rendle S, Schmidt-Thieme L (2010) Pairwise interaction tensor factorization for personalized tag recommendation[C]//proceedings of the third ACM international conference on web search and data mining. ACM:81–90

  13. Juan Y, et al. (2016) "Field-aware factorization machines for CTR prediction." Proceedings of the 10th ACM Conference on Recommender Systems. ACM

  14. He X, et al. (2014) "Practical lessons from predicting clicks on ads at facebook." Proceedings of the Eighth International Workshop on Data Mining for Online Advertising. ACM

  15. LeCun Y, Bengio Y, Hinton G (2015) "Deep learning." Nature 521.7553: 436

    Article  Google Scholar 

  16. Chen LC, Papandreou G, Kokkinos I et al (2018) Deeplab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected crfs[J]. IEEE Trans Pattern Anal Mach Intell 40(4):834–848

    Article  Google Scholar 

  17. Amodei D, Ananthanarayanan S, Anubhai R, et al. (2016) Deep speech 2: End-to-end speech recognition in english and mandarin[C]//International conference on machine learning: 173–182

  18. Young T, Hazarika D, Poria S et al (2018) Recent trends in deep learning based natural language processing[J]. IEEE Comput Intell Mag 13(3):55–75

    Article  Google Scholar 

  19. Gao H, Kong D, Lu M et al (2018) Attention convolutional neural network for advertiser-level click-through rate forecasting[C]//proceedings of the 2018 world wide web conference on world wide web. International World Wide Web Conferences Steering Committee:1855–1864

  20. Zhang W, Du T, Wang J (2016) Deep learning over multi-field categorical data[C]//European conference on information retrieval. Springer, Cham, pp 45–57

    Google Scholar 

  21. Liu Q, Yu F, Wu S et al (2015) A convolutional click prediction model[C]//proceedings of the 24th ACM international on conference on information and knowledge management. ACM:1743–1746

  22. Zhang Y, Dai H, Xu C, et al. (2014) Sequential Click Prediction for Sponsored Search with Recurrent Neural Networks[C]//AAAI. 14: 1369–1375

  23. Qu Y, Cai H, Ren K et al (2016) Product-based neural networks for user response prediction[C]//2016 IEEE 16th international conference on data mining (ICDM). IEEE:1149–1154

  24. Shan Y, Hoens TR, Jiao J et al (2016) Deep crossing: web-scale modeling without manually crafted combinatorial features[C]//proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining. ACM:255–262

  25. Zagoruyko S, Komodakis N (2016) Wide residual networks[J]. arXiv preprint arXiv:1605.07146

  26. Wang R, Fu B, Fu G et al (2017) Deep & cross network for ad click predictions[C]//proceedings of the ADKDD'17. ACM 12

  27. Guo H, Tang R, Ye Y, et al. (2018) DeepFM: An End-to-End Wide & Deep Learning Framework for CTR Prediction[J]. arXiv preprint arXiv:1804.04950

  28. Xiao J, Ye H, He X, et al. (2017) Attentional factorization machines: Learning the weight of feature interactions via attention networks[J]. arXiv preprint arXiv:1708.04617

  29. Vaswani A, Shazeer N, Parmar N, et al. (2017) Attention is all you need[C]//Advances in neural information processing systems: 5998–6008

  30. Wang Q, Liu F, Xing S et al (2018) A new approach for advertising CTR prediction based on deep neural network via attention mechanism[J]. Comput Math Method Med 2018

  31. Ghasemi F, Mehridehnavi A, Fassihi A et al (2018) Deep neural network in QSAR studies using deep belief network[J]. Appl Soft Comput 62:251–258

    Article  Google Scholar 

  32. Zhou G, Zhu X, Song C et al (2018) Deep interest network for click-through rate prediction[C]//proceedings of the 24th ACM SIGKDD international conference on Knowledge Discovery & Data Mining. ACM:1059–1068

  33. Chung J, Gulcehre C, Cho KH, et al. (2014) Empirical evaluation of gated recurrent neural networks on sequence modeling[J]. arXiv preprint arXiv:1412.3555

  34. Hochreiter S, Schmidhuber J (1997) Long short-term memory[J]. Neural Comput 9(8):1735–1780

    Article  Google Scholar 

  35. Hidasi B, Karatzoglou A, Baltrunas L, et al. (2015) Session-based recommendations with recurrent neural networks[J]. arXiv preprint arXiv:1511.06939

  36. Vincent P, Larochelle H, Bengio Y, et al. Extracting and composing robust features with denoising autoencoders[C]//Proceedings of the 25th international conference

  37. Baltrunas L, et al. (2015) "Frappe: Understanding the usage and perception of mobile app recommendations in-the-wild." arXiv preprint arXiv:1505.03014

  38. Harper FM, Konstan JA (2016) "The movielens datasets: History and context." ACM Transactions on Interactive Intelligent Systems (TiiS) 5.4: 19

    Article  Google Scholar 

  39. Li L, Greene T, Hu B (2018) A simple method to estimate the time-dependent receiver operating characteristic curve and the area under the curve with right censored data[J]. Stat Methods Med Res 27(8):2264–2278

    Article  MathSciNet  Google Scholar 

  40. Zhang L, Shen W, Huang J et al (2019) Field-aware neural factorization machine for click-through rate prediction[J]. IEEE Access 7:75032–75040

    Article  Google Scholar 

  41. Holzinger A (2018) "From machine learning to explainable AI." 2018 World Symposium on Digital Intelligence for Systems and Machines (DISA). IEEE

Download references

Acknowledgments

This work was supported by the following grants: National Natural Science Foundation of China (61572300,81871508,61773246); Taishan Scholar Program of Shandong Province of China (TSHW201502038); Natural Science Foundation of Shandong Province (ZR2018ZB0419); Primary Research and Development Plan of Shandong Province (2017GGX10112).

Author information

Authors and Affiliations

  1. School of Information Science & Engineering, Shandong Normal University, Jinan, China

    Hao Li, Huichuan Duan, Yuanjie Zheng, Qianqian Wang & Yu Wang

  2. Key Lab of Intelligent Computing and Information Security in Universities of Shandong, Shandong Provincial Key Laboratory for Novel Distributed Computer Software Technology, Institute of Biomedical Sciences, Shandong Normal University, Jinan, 250358, China

    Yuanjie Zheng

Authors
  1. Hao Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huichuan Duan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuanjie Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Qianqian Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yu Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huichuan Duan.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Duan, H., Zheng, Y. et al. A CTR prediction model based on user interest via attention mechanism. Appl Intell 50, 1192–1203 (2020). https://doi.org/10.1007/s10489-019-01571-9

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01571-9

Keywords

Search

Navigation