Skip to main content

Advertisement

Springer Nature Link
Log in

Graph relation embedding network for click-through rate prediction

  • Regular Paper
  • Published:
Knowledge and Information Systems Aims and scope Submit manuscript

Abstract

Most deep click-through rate (CTR) prediction models utilize a mainstream framework, which consists of the embedding layer and the feature interaction layer. Embeddings rich in semantic information directly benefit the downstream frameworks to mine potential information and achieve better performance. However, the embedding layer is rarely optimized in the CTR field. Although mapped into a low-dimensional embedding space, discrete features are still sparse. To solve this problem, we build graph structures to mine the similar interest of users and the co-occurrence relationship of items from click behavior sequences, and regard them as prior information for embedding optimization. For interpretable graph structures, we further propose graph relation embedding networks (GREENs), which utilize adapted order-wise graph convolution to alleviate the problems of data sparsity and over-smoothing. Moreover, we also propose a graph contrastive regularization module, which further normalizes graph embedding by maintaining certain graph structure information. Extensive experiments have proved that by introducing our embedding optimization methods, significant performance improvement is achieved.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Explore related subjects

Discover the latest articles and news from researchers in related subjects, suggested using machine learning.

Notes

  1. http://jmcauley.ucsd.edu/data/amazon/.

  2. https://grouplens.org/datasets/movielens/20m/.

References

  1. Bo D, Wang X, Shi C, Zhu M, Lu E, Cui P (2020) Structural deep clustering network. In: Proceedings of the Web conference 2020, pp 1400–1410

  2. Chen H, Yin H, Sun X, Chen T, Gabrys B, Musial K (2020) Multi-level graph convolutional networks for cross-platform anchor link prediction. arXiv preprint arXiv:2006.01963

  3. Cheng H-T, Koc L, Harmsen J, Shaked T, Chandra T, Aradhye H, Anderson G, Corrado G, Chai W, Ispir M et al (2016) Wide & deep learning for recommender systems. In: Proceedings of the 1st workshop on deep learning for recommender systems, pp 7–10

  4. Cui G, Zhou J, Yang C, Liu Z (2020) Adaptive graph encoder for attributed graph embedding. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 976–985

  5. Fan W, Ma Y, Li Q, He Y, Zhao E, Tang J, Yin D (2019) Graph neural networks for social recommendation. In: The World Wide Web conference, pp 417–426

  6. Fawcett T (2006) An introduction to roc analysis. Pattern Recognit Lett 27(8):861–874

    Article  MathSciNet  Google Scholar 

  7. Guo H, Tang R, Ye Y, Li Z, He X (2017) Deepfm: a factorization-machine based neural network for ctr prediction. arXiv preprint arXiv:1703.04247

  8. Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

  9. Harper FM, Konstan JA (2015) The movielens datasets: history and context. ACM Trans Interact Intell Syst (tiis) 5(4):1–19

    Google Scholar 

  10. He R, McAuley J (2016) Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In: Proceedings of the 25th international conference on world wide web, pp 507–517

  11. He X, Deng K, Wang X, Li Y, Zhang Y, Wang M (2020) Lightgcn: simplifying and powering graph convolution network for recommendation. arXiv preprint arXiv:2002.02126

  12. Hjelm RD, Fedorov A, Lavoie-Marchildon S, Grewal K, Bachman P, Trischler A, Bengio Y (2018) Learning deep representations by mutual information estimation and maximization. arXiv preprint arXiv:1808.06670

  13. Juan Y, Zhuang Y, Chin W-S, Lin C-J (2016) Field-aware factorization machines for ctr prediction. In: Proceedings of the 10th ACM conference on recommender systems, pp 43–50

  14. Kim K, Kwon E, Park J (2021) Deep user segment interest network modeling for click-through rate prediction of online advertising. IEEE Access 9:9812–9821

    Article  Google Scholar 

  15. Kipf TN, Welling M (2016a) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

  16. Kipf TN, Welling M (2016b) Variational graph auto-encoders. arXiv preprint arXiv:1611.07308

  17. Li D, Liu H, Zhang Z, Lin K, Fang S, Li Z, Xiong NN (2022) Carm: Confidence-aware recommender model via review representation learning and historical rating behavior in the online platforms. Neurocomputing 455:283–296

    Article  Google Scholar 

  18. Li F, Chen Z, Wang P, Ren Y, Zhang D, Zhu X (2019) Graph intention network for click-through rate prediction in sponsored search. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 961–964

  19. Li X, Hu Y, Sun Y, Hu J, Zhang J, Qu M (2020) A deep graph structured clustering network. IEEE Access

  20. Li Z, Liu H, Zhang Z, Liu T, Xiong NN (2021) Learning knowledge graph embedding with heterogeneous relation attention networks. IEEE Trans Neural Netw Learn Syst

  21. Liu H, Fang S, Zhang Z, Li D, Lin K, Wang J (2021) Mfdnet: collaborative poses perception and matrix fisher distribution for head pose estimation. IEEE Trans Multim

  22. Liu H, Zheng C, Li D, Shen X, Lin K, Wang J, Zhang Z, Zhang Z, Xiong NN (2020) Edmf: efficient deep matrix factorization with review feature learning for industrial recommender system. IEEE Trans Ind Inform

  23. Liu T, Liu H, Li Y-F, Chen Z, Zhang Z, Liu S (2019) Flexible ftir spectral imaging enhancement for industrial robot infrared vision sensing. IEEE Trans Ind Inform 16(1):544–554

    Article  Google Scholar 

  24. Liu T, Liu H, Li Y, Zhang Z, Liu S (2018) Efficient blind signal reconstruction with wavelet transforms regularization for educational robot infrared vision sensing. IEEE/ASME Trans Mechatron 24(1):384–394

    Article  Google Scholar 

  25. Mnih A, Kavukcuoglu K (2013) Learning word embeddings efficiently with noise-contrastive estimation. Adv Neural Inform Process Syst 26:2265–2273

    Google Scholar 

  26. Pironkov G, Dupont S, Dutoit T (2016) Speaker-aware long short-term memory multi-task learning for speech recognition. In: 2016 24th European signal processing conference (EUSIPCO). IEEE, pp 1911–1915

  27. Qu Y, Cai H, Ren K, Zhang W, Yu Y, Wen Y, Wang J (2016) Product-based neural networks for user response prediction. In: 2016 IEEE 16th international conference on data mining (ICDM). IEEE, pp 1149–1154

  28. Rendle S, Schmidt-Thieme L (2010) Pairwise interaction tensor factorization for personalized tag recommendation. In: Proceedings of the third ACM international conference on Web search and data mining’, pp 81–90

  29. Richardson M, Dominowska E, Ragno R (2007) Predicting clicks: estimating the click-through rate for new ads. In: Proceedings of the 16th international conference on World Wide Web, pp 521–530

  30. Shan Y, Hoens TR, Jiao J, Wang H, Yu D, Mao J (2016) Deep crossing: Web-scale modeling without manually crafted combinatorial features. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 255–262

  31. Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio, Y (2017) ‘Graph attention networks. arXiv preprint arXiv:1710.10903

  32. Veličković P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2018) Deep graph infomax. arXiv preprint arXiv:1809.10341

  33. Wang X, He X, Wang M, Feng F, Chua T-S (2019) Neural graph collaborative filtering. In: Proceedings of the 42nd international ACM SIGIR conference on research and development in information retrieval, pp 165–174

  34. Wu F, Zhang T, Souza Jr, AHd, Fifty C, Yu T, Weinberger KQ (2019) Simplifying graph convolutional networks. arXiv preprint arXiv:1902.07153

  35. Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? arXiv preprint arXiv:1810.00826

  36. Yan L, Li W-J, Xue G-R, Han D (2014) Coupled group lasso for web-scale ctr prediction in display advertising. In: International conference on machine learning, pp 802–810

  37. Yi B, Shen X, Liu H, Zhang Z, Zhang W, Liu S, Xiong N (2019) Deep matrix factorization with implicit feedback embedding for recommendation system. IEEE Trans Ind Inf 15(8):4591–4601

    Article  Google Scholar 

  38. Zhang H, Ji Y, Li J, Ye Y (2015) A triple wing harmonium model for movie recommendation. IEEE Trans Ind Inf 12(1):231–239

    Google Scholar 

  39. Zhang J, Shi X, Zhao S, King I (2019) Star-gcn: Stacked and reconstructed graph convolutional networks for recommender systems. arXiv preprint arXiv:1905.13129

  40. Zhang X, Wang Z, Du B (2021) Deep dynamic interest learning with session local and global consistency for click-through rate predictions. IEEE Trans Ind Inf

  41. Zhang Z, Li Z, Liu H, Xiong NN (2020) Multi-scale dynamic convolutional network for knowledge graph embedding. IEEE Trans Knowl Data Eng

  42. Zhao WX, Li S, He Y, Chang EY, Wen J-R, Li X (2015) Connecting social media to e-commerce: Cold-start product recommendation using microblogging information. IEEE Trans Knowl Data Eng 28(5):1147–1159

    Article  Google Scholar 

  43. Zhou G, Mou N, Fan Y, Pi Q, Bian W, Zhou C, Zhu X, Gai K (2019) Deep interest evolution network for click-through rate prediction. In: Proceedings of the AAAI conference on artificial intelligence, vol 33, pp 5941–5948

  44. Zhou G, Zhu X, Song C, Fan Y, Zhu H, Ma X, Yan Y, Jin J, Li H, Gai K (2017) Deep interest network for click-through rate prediction. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1059–1068

  45. Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2018) Graph neural networks: a review of methods and applications. arXiv preprint arXiv:1812.08434

  46. Zhu H, Jin J, Tan C, Pan F, Zeng Y, Li H, Gai K (2017) Optimized cost per click in taobao display advertising. In: Proceedings of the 23rd ACM SIGKDD international conference on knowledge discovery and data mining, pp 2191–2200

Download references

Acknowledgements

This work was partially supported by the Key Research Project of Zhejiang Province (No. 2022C01145) and the National Science Foundation of China (No. U20A20173 and No. 62125206).

Author information

Authors and Affiliations

  1. Zhejiang University, Hangzhou, China

    Yixuan Wu & Shuiguang Deng

  2. Shandong University, Weihai, China

    Youpeng Hu, Xunkai Li & Ronghui Guo

  3. Nanjing University, Nanjing, China

    Xin Xiong

Authors
  1. Yixuan Wu
    View author publications

    You can also search for this author inPubMed Google Scholar

  2. Youpeng Hu
    View author publications

    You can also search for this author inPubMed Google Scholar

  3. Xin Xiong
    View author publications

    You can also search for this author inPubMed Google Scholar

  4. Xunkai Li
    View author publications

    You can also search for this author inPubMed Google Scholar

  5. Ronghui Guo
    View author publications

    You can also search for this author inPubMed Google Scholar

  6. Shuiguang Deng
    View author publications

    You can also search for this author inPubMed Google Scholar

Corresponding author

Correspondence to Shuiguang Deng.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wu, Y., Hu, Y., Xiong, X. et al. Graph relation embedding network for click-through rate prediction. Knowl Inf Syst 64, 2543–2564 (2022). https://doi.org/10.1007/s10115-022-01714-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10115-022-01714-4

Keywords