Abstract
Click-through rate (CTR) prediction is a crucial task for personalized services such as online advertising and recommender system. Many methods including Factorization Machines (FM) and complex deep neural models have been proposed to predict CTR and achieve good results. However, they usually optimize the parameters through a global objective function such as minimizing logloss and mean square error for all training samples. Obviously they intend to capture global knowledge of user click behavior, but ignore local information. Therefore, we propose a novel approach of Retrieval-based Factorization Machines (RFM) for CTR prediction, which enhances FM by the neighbor-based local information. During online testing, we also leverage the K-Means clustering technique to partition the large training set to multiple small regions for efficient retrieval of neighbors. We evaluate our RFM model on three public datasets. The experimental results show that RFM performs better than existing models including FM and deep neural models, and is efficient because of the small number of model parameters.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
McMahan, H.B., et al.: Ad click prediction: a view from the trenches. In: SIGKDD, pp. 1222–1230 (2013)
Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009)
He, X., et al.: Practical lessons from predicting clicks on ads at facebook. In: ADKDD@KDD, pp. 5:1–5:9
Chang, Y.-W., Hsieh, C.-J., Chang, K.-W., Ringgaard, M., Lin, C.-J.: Training and testing low-degree polynomial data mappings via linear SVM. J. Mach. Learn. Res. 11, 1471–1490 (2010)
Rendle, S.: Factorization machines. In: ICDM, pp. 995–1000. IEEE Computer Society (2010)
Juan, Y.-C., Zhuang, Y., Chin, W.-S.: Field-aware factorization machines for CTR prediction. In: RecSys, pp. 43–50
Blondel, M., Fujino, A., Ueda, N., Ishihata, M.: Higher-order factorization machines. In: NIPS, pp. 3351–3359 (2016)
Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for CTR prediction. In: IJCAI, pp. 1725–1731. ijcai.org (2017)
He, X., Chua, T.-S.: Neural factorization machines for sparse predictive analytics. In: SIGIR, pp. 355–364
Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., Chua, T.-S.: Attentional factorization machines: learning the weight of feature interactions via attention networks. In: IJCAI, pp. 3119–3125. AAAI Press (2017)
Cheng, H.-T., et al.: Wide & deep learning for recommender systems. In: DLRS@RecSys, pp. 7–10. ACM (2016)
Shan, Y., Hoens, T.R., Jiao, J., Wang, H., Yu, D., Mao, J.C.: Deep crossing: web-scale modeling without manually crafted combinatorial features. In: KDD, pp. 255–262. ACM (2016)
Tao, Z., Wang, X., He, X., Huang, X., Chua, T.-S.: Hoafm: A high-order attentive factorization machine for CTR prediction. Inf. Process. Manag. 57(6), 102076 (2020)
Qu, Y., et al.: Product-based neural networks for user response prediction over multi-field categorical data. ACM Trans. Inf. Syst., 5:1–5:35 (2019)
Ebesu, T., Shen, B., Fang, Y.: Collaborative memory network for recommendation systems. In: SIGIR, pp. 515–524. ACM (2018)
Coates, A., Ng, A.Y.: Learning feature representations with K-Means. In: Montavon, G., Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 7700, pp. 561–580. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-35289-8_30
Ioffe, S., Szegedy, C.: Batch normalization: Accelerating deep network training by reducing internal covariate shift. In: ICML. JMLR Workshop and Conference Proceedings, vol. 37, pp. 448–456. JMLR.org (2015)
Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Baltrunas, L., Church, K., Karatzoglou, A., Oliver, N.: Frappe: understanding the usage and perception of mobile app recommendations in-the-wild. CoRR, abs/1505.03014 (2015)
Harper, F.M., Konstan, J.A.: The movielens datasets: history and context. ACM Trans. Interactive Intell. Syst. (tiis), 5(4), 1–19 (2015)
Duchi, J.C., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization. J. Mach. Learn. Res. 12, 2121–2159 (2011)
Jain, A.K.: Data clustering: 50 years beyond k-means. Pattern Recognit. Lett. 31(8), 651–666 (2010)
Gunopulos, D.: Encyclopedia of Database Systems, 2nd edn. Springer, New York (2018). https://doi.org/10.1007/978-1-4614-8265-9
Acknowledgements
This work was supported by National Natural Science Foundation of China (No. 62072017, 61702024).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, X., Huang, Y., Zhao, X., Zhao, W., Tang, Y., Duan, Y. (2021). Retrieval-Based Factorization Machines for CTR Prediction. In: Zhang, W., Zou, L., Maamar, Z., Chen, L. (eds) Web Information Systems Engineering – WISE 2021. WISE 2021. Lecture Notes in Computer Science(), vol 13081. Springer, Cham. https://doi.org/10.1007/978-3-030-91560-5_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-91560-5_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-91559-9
Online ISBN: 978-3-030-91560-5
eBook Packages: Computer ScienceComputer Science (R0)