Abstract
Click-through-rate (CTR) prediction is a crucial task in recommendation systems. The accuracy of CTR prediction is strongly influenced by the precise extraction of essential data and the modeling strategy chosen. The data of the CTR task are often very sparse, and Factorization Machines (FMs) are a class of general predictors working effectively with it. However, the performance of FMs can be limited by the fixed feature representation and the same weight of different features. In this work, we propose an improved Bitwise Feature Importance Factorization Machine (BFIFM) to improve the accuracy. The necessity of learning the degree of effect of the same feature under various situations is learned through the low-order intersection method, and the deep neural network (DNN) in our model is used in parallel to study high-order intersections. According to the final results obtained, the BFIFM model significantly outperforms other state-of-the-art models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Gortmaker, S.L., Hosmer, D.W., Lemeshow, S.: Applied logistic regression. Contemp. Sociol. 23(1), 159 (1994)
Rendle, S.: Factorization machines. In: ICDM 2010, The 10th IEEE International Conference on Data Mining. IEEE (2010)
Vovk, V., et al.: Erratum to: The Fundamental Nature of the Logloss Function (2015)
Singh, G., Sachan, M.: Multilayer perceptron (MLP) neural network technique for offline handwritten Gurmukhi character recognition. In: IEEE International Conference on Computational Intelligence Computing Research. IEEE (2015)
Juan, Y., Zhuang, Y., Chin, W.-S., Lin, C.-J.: Field-aware factorization machines for CTR prediction. In: Proceedings of the 10th ACM Conference on Recommender Systems, pp. 43–50. ACM (2016)
Zhang, W., Du, T., Wang, J.: Deep learning over multi-field categorical data. In: Ferro, N., et al. (eds.) ECIR 2016. LNCS, vol. 9626, pp. 45–57. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-30671-1_4
Cheng, H.T., et al.: Wide & Deep Learning for Recommender Systems. In: Proceedings of the 1st Workshop on Deep Learning for Recommender Systems (DLRS 2016), pp. 7–10. Association for Computing Machinery, New York, NY, USA
Martins, A., Astudillo, R.F.: From softmax to sparsemax: a sparse model of attention and multilabel classification. In: Proceedings of the 33rd International Conference on International Conference on Machine Learning, vol. 48, pp. 1614–1623 (ICML’16) (2016) JMLR.org
He, X., Chua, T.S.: Neural factorization machines for sparse predictive analytics. ACM SIGIR Forum 51(cd), 355–364 (2017)
Rivera-Trigueros, I.: Machine translation systems and quality assessment: a systematic review. Lang. Resour. Eval. 56, 593–619 (2021)
Xiao, J., Ye, H., He, X., Zhang, H., Wu, F., Chua, T. S.: Attentional factorization machines: learning the weight of feature interactions via attention networks: In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence (IJCAI-17), pp. 3119–3125 (2017)
Guo, H., Tang, R., Ye, Y., Li, Z., He, X.: DeepFM: A factorization-machine based neural network for CTR prediction. In: Twenty-Sixth International Joint Conference on Artificial Intelligence (2017)
Wang, R., Fu, B., Fu, G., Wang, M.: Deepcross network for ad click predictions. In: ACM (2017)
Lian, J., Zhou, X., Zhang, F., Chen, Z., Xie, X., Sun, G.: xDeepfm: combining explicit and implicit feature interactions for recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, 19–23 Aug 2018
Song, W., Shi, C., Xiao, Z., Duan, Z., Jian, T.: Autoint: automatic feature interaction learning via self-attentive neural networks. In: Proceedings of the 28th ACM International Conference on Information and Knowledge Management (CIKM’19), pp. 1161–1170. Association for Computing Machinery, New York, NY, USA (2018)
Yu, Y., Wang, Z., Yuan, B.: An input-aware factorization machine for sparse prediction. In: Twenty-Eighth International Joint Conference on Artificial Intelligence IJCAI-19 (2019)
Huang, T., Zhiqi, Z., Junlin, Z.: FiBiNET: combining feature importance and bilinear feature interaction for click-through rate prediction. In: Proceedings of the 13th ACM Conference on Recommender Systems, pp. 169–177 (2019)
Lu, W., Yu, Y., Chang, Y., Wang, Z., Yuan, B.: A dual input-aware factorization machine for CTR prediction. In: Twenty-Ninth International Joint Conference on Artificial Intelligence and Seventeenth Pacific Rim International Conference on Artificial Intelligence IJCAI-PRICAI-20 (2020)
Ruiqin, W., Zongda, W., Yunliang, J., Jungang, L.: An integrated recommendation model based on two-stage deep learning. J. Comput. Res. Dev. 56, 1661 (2019)
Alhijawi, B., Kilani, Y.: The recommender system: a survey. Int. J. Adv. Intell. Paradigms 15(3), 1 (2020)
Baldi, P., Sadowski, P.: Understanding dropout. Adv. Neural Inform. Process. Syst. 26(1) (2013)
Li, Y., Yuan, Y.: Convergence analysis of two-layer neural networks with ReLU activation (2017)
Kingma, D., Ba, J.: Adam: a method for stochastic optimization. Comput. Sci. (2014)
Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. In: 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, Workshop Track Proceedings 2–4 May 2013
Barkan, O., Koenigstein, N.: Item2vec: neural item embedding for collaborative filtering. IEEE (2016)
Vaswani, A., et al.: Attention is all you need. In: 31st Conference on Neural Information Processing Systems (NIPS 2017), Long Beach, CA, USA (2017)
Batmaz, Z., Yurekli, A., Bilge, A., Kaleli, C.: A review on deep learning for recommender systems: challenges and remedies. Artif. Intell. Rev. 52(1), 1–37 (2018)
Kaggle Science Community: Display advertising challenge: Predict click-through rates on display ads. https://www.kaggle.com/c/criteo-display-ad-challeng (2014)
Kaggle Science Community: Click-through rate prediction: predict whether a mobile ad will be clicked. https://www.kaggle.com/c/avazu-ctr-prediction (2015)
Acknowledgment
This work is supported by Hainan Province Science and Technology Special Fund, which is Research and Application of Intelligent Recommendation Technology Based on Knowledge Graph and User Portrait (No.ZDYF2020039). Thanks to Professor CaiMao Li, the correspondent of this paper.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Li, H., Li, C., Hou, Y., Lin, H., Chen, Q. (2022). Factorization Machine Based on Bitwise Feature Importance for CTR Prediction. In: Wang, Y., Zhu, G., Han, Q., Wang, H., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2022. Communications in Computer and Information Science, vol 1628. Springer, Singapore. https://doi.org/10.1007/978-981-19-5194-7_3
Download citation
DOI: https://doi.org/10.1007/978-981-19-5194-7_3
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-5193-0
Online ISBN: 978-981-19-5194-7
eBook Packages: Computer ScienceComputer Science (R0)