Abstract
Factorization-based methods, which can automatically model second-order or higher-order cross features, have been the benchmark models for click-through rate (CTR) prediction. In general, they enumerate all cross features with a predetermined order and filter out useless interactions through model training. However, two significant challenges remain. First, a maximum order for feature interactions needs to be defined in advance, imposing a trade-off between computational cost and the expression ability of models; Second, enumerating all feature interactions may introduce unwanted noise. In this work, we propose a novel model called Input Enhanced Logarithmic Factorization Network (ILFN), which can effectively learn arbitrary-order feature interactions and identify useful cross features. More importantly, ILFN can take full advantage of the original feature distribution in a discriminative way.
The core of ILFN is the Input Enhanced Component (IEC), which can represent the impact of each feature in a feature interaction as a trainable coefficient to learn arbitrary-order cross features. Moreover, IEC can reduce the demand for logarithmic neurons by exploiting the essential raw information and does not need to incorporate the deep neural network (DNN) to model high-order interactions. Therefore, ILFN is more effective and efficient and can converge to satisfactory results faster. Extensive experiments on four real-world datasets demonstrate that our ILFN model can outperform start-of-the-art methods. The effectiveness of each proposed component is also verified by hyper-parameter and ablation studies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The models in bold are baselines in the experimental part.
- 2.
- 3.
- 4.
- 5.
- 6.
References
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: ICLR (2015)
Wang, R., Shivanna, R., et al.: DCN V2: improved deep & cross network and practical lessons for web-scale learning to rank systems. In: WWW, pp. 1785–1797 (2021)
Cheng, H.-T., Koc, L., et al.: Wide & deep learning for recommender systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems, pp. 7–10 (2016)
Cheng, W., Shen, Y., Huang, L.: Adaptive factorization network: learning adaptive-order feature interactions. In: AAAI, pp. 3609–3616 (2020)
McMahan, H.B., Holt, G., et al.: Ad click prediction: a view from the trenches. In: The 19th SIGKDD, 11–14 August 2013 (2013)
Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: a factorization-machine based neural network for CTR prediction. In: IJCAI, pp. 1725–1731 (2017)
Graepel, T., Candela, J.Q., Borchert, T., Herbrich, R.: Web-scale Bayesian click-through rate prediction for sponsored search advertising in Microsoft’s Bing search engine. Omnipress (2010)
Wang, Z., Zhang, R., Qi, J., Yuan, B.: DBSVEC: density-based clustering using support vector expansion. In: ICDE, pp. 280–291. IEEE (2019)
Lee, K., Orten, B., Dasdan, A., Li, W.: Estimating conversion rate in display advertising from past erformance data. In: KDD, pp. 768–776 (2012)
Wang, Z., Liu, L., Tao, D.: Deep streaming label learning. In: International Conference on Machine Learning (ICML) (2020)
Qu, Y., et al.: Product-based neural networks for user response prediction. In: ICDM, pp. 1149–1154. IEEE (2016)
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., Sun, G.: xDeepFM: combining explicit and implicit feature interactions for recommender systems. In: KDD, pp. 1754–1763 (2018)
He, X., Chua, T.-S.: Neural factorization machines for sparse predictive analytics. In: SIGIR, pp. 355–364 (2017)
Xiao, J., Ye, H., He, X., Zhang, H., et al.: Attentional factorization machines: learning the weight of feature interactions via attention networks. In: IJCAI (2017)
Hines, J.W.: A logarithmic neural network architecture for unbounded non-linear function approximation. In: IEEE ICNN 1996 (1996)
Juan, Y., Zhuang, Y., Chin, W.-S., Lin, C.-J.: Field-aware factorization machines for CTR prediction. In: RecSys (2016)
Song, W., Shi, C., et al.: AutoInt: automatic feature interaction learning via self-attentive neural networks. In: CIKM (2019)
Wang, R., Fu, B., Fu, G., Wang, M.: Deep & cross network for ad click predictions. In: KDD, pp. 1–7 (2017)
Huang, T., Zhang, Z., Zhang, J.: FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In: ACM RecSys (2019)
Rendle, S.: Factorization machines. In: 2010 IEEE International Conference on Data Mining, pp. 995–1000. IEEE (2010)
Blondel, M., Fujino, A., Ueda, N., Ishihata, M.: Higher-order factorization machines. In: NIPS (2016)
Yu, Y., Wang, Z., Yuan, B.: An input-aware factorization machine for sparse prediction. In: IJCAI (2019)
Zhang, W., Du, T., Wang, J.: Deep learning over multi-field categorical data. In: Ferro, N., et al. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 45–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_4
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, X., Wang, Z., Wu, X., Yuan, B., Wang, X. (2022). Input Enhanced Logarithmic Factorization Network for CTR Prediction. In: Gama, J., Li, T., Yu, Y., Chen, E., Zheng, Y., Teng, F. (eds) Advances in Knowledge Discovery and Data Mining. PAKDD 2022. Lecture Notes in Computer Science(), vol 13282. Springer, Cham. https://doi.org/10.1007/978-3-031-05981-0_4
Download citation
DOI: https://doi.org/10.1007/978-3-031-05981-0_4
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-05980-3
Online ISBN: 978-3-031-05981-0
eBook Packages: Computer ScienceComputer Science (R0)