Abstract
In the Internet, categorical features are high-dimensional and sparse, and to obtain its low-dimensional and dense representation, the embedding mechanism plays an important role in the click-through rate prediction of the recommendation system. Prior works have proved that residual network is helpful to improve the performance of deep learning models, but there are few works to learn and optimize the embedded representation of raw features through residual thought in recommendation systems. Therefore, we designed a self-residual embedding structure to learn the distinction between the randomly initialized embedding vector and the ideal embedding vector by calculating the self-correlation score, and applied it to our proposed SRFM model. Extensive experiments on four real datasets show that the SRFM model can achieve satisfactory performance compared with the superior model. Also, the self-residual embedding mechanism can improve the prediction performance of some existing deep learning models to a certain extent.
Supported by National Natural Science Foundation of China (61702059, 61962038), Guangxi Bagui Teams for Innovation and Research (201979), Science and Technology Development Fund Macau (SKL-IOTSC-2021-2023), University of Macau (MYRG2019-00119-FST).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Cheng, H.T., et al.: Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pp. 7–10 (2016)
Cheng, W., Shen, Y., Huang, L.: Adaptive factorization network: learning adaptive-order feature interactions. In: Proceedings of the AAAI Conference on Artificial Intelligence vol. 34, no. (04), pp. 3609–3616 (2020)
Hand, D.J., Till, R.J.: A simple generalisation of the area under the roc curve for multiple class classification problems. Mach. Learn. 45, 171–186 (2001)
Graepel, T., Candela, J.Q., Borchert, T., Herbrich, R.: Web-scale Bayesian click-through rate prediction for sponsored search advertising in microsoft’s bing search engine. In: Proceedings of the 27th International Conference on Machine Learning (ICML-10), 21–24 June 2010, Haifa, Israel, pp. 13–20 (2010)
Han, Y., Gu, P., Gao, W., Xu, G., Wu, J.: Aspect-level sentiment capsule network for micro-video click-through rate prediction. World Wide Web, 1–20 (2021)
He, X., Chua, T.S.: Neural factorization machines for sparse predictive analytics. In: Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 355–364 (2017)
Huang, F., Li, X., Yuan, C., Zhang, S., Zhang, J., Qiao, S.: Attention-emotion-enhanced convolutional LSTM for sentiment analysis. IEEE Trans. Neural Netw. Learn. Syst., 1–14 (2021). https://doi.org/10.1109/TNNLS.2021.3056664
Huang, T., She, Q., Wang, Z., Zhang, J.: GateNet: gating-enhanced deep network for click-through rate prediction (2020)
Huifeng, G., Ruiming, T., Yunming, Y., Zhenguo, L., Xiuqiang, H.: DeepFM: a factorization-machine based neural network for CTR prediction. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 1725–1731. AAAI Press (2017)
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on Machine Learning, vol. 37, pp. 448–456 (2015)
Juan, Y., Zhuang, Y., Chin, W.S., Lin, C.J.: Field-aware factorization machines for CTR prediction. In: Proceedings of the 10th ACM Conference on Recommender Systems. ACM, September 2016
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Comput. Sci. (2014)
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., Sun, G.: xDeepFM: combining explicit and implicit feature interactions for recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1754–1763 (2018)
Liu, B., et al.: AutoFIS: automatic feature interaction selection in factorization models for click-through rate prediction. In: Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM (2020)
Luo, Y., et al.: AutoCross: automatic feature crossing for tabular data in real-world applications. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1936–1945 (2019)
Park, H.-A.: An introduction to logistic regression. J. Korean Acad. Nurs. 43(2), 154–164 (2013)
Qu, Y., et al.: Product-based neural networks for user response prediction. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 1149–1154 (2016)
Rendle, S.: Factorization machines. In: 2010 IEEE 10th International Conference on Data Mining (ICDM), pp. 995–1000 (2010)
Sheng, X.R., et al.: One model to serve all: star topology adaptive recommender for multi-domain CTR prediction (2021)
Song, W., et al.: AutoInt: automatic feature interaction learning via self-attentive neural networks. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management. ACM, November 2019
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Vaswani, A., et al.: Attention is all you need. CoRR abs/1706.03762 (2017)
Wang, R., Fu, B., Fu, G., Wang, M.: Deep & cross network for ad click predictions. In: ADKDD 2017 (2017)
Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., Chua, T.: Attentional factorization machines: learning the weight of feature interactions via attention networks. CoRR abs/1708.04617 (2017)
Zhang, W., Du, T., Wang, J.: Deep learning over multi-field categorical data. In: Ferro, N., et al. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 45–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_4
Zhao, Z., Fang, Z., Peng, C., Bao, Y., Yan, W.: Dimension relation modeling for click-through rate prediction. In: The 29th ACM International Conference on Information and Knowledge Management, CIKM 2020 (2020)
Zhou, G., Wu, K., Bian, W., Yang, Z., Zhu, X., Gai, K.: Res-embedding for deep learning based click-through rate prediction modeling. In: The 1st International Workshop (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Sun, J., Yin, Y., Huang, F., Zhou, M., U, L.H. (2021). Self-residual Embedding for Click-Through Rate Prediction. In: U, L.H., Spaniol, M., Sakurai, Y., Chen, J. (eds) Web and Big Data. APWeb-WAIM 2021. Lecture Notes in Computer Science(), vol 12859. Springer, Cham. https://doi.org/10.1007/978-3-030-85899-5_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-85899-5_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-85898-8
Online ISBN: 978-3-030-85899-5
eBook Packages: Computer ScienceComputer Science (R0)